Systems, methods, and computer program products for home and landscape design

ABSTRACT

This patent application relates generally to systems, methods, and computer program products for home and/or landscape design.

CROSS-REFERENCE TO RELATED APPLICATIONS

This patent application claims priority to and is a continuationapplication of U.S. patent application Ser. No. 12/270,470, entitled“Systems, Methods, And Computer Program Products For Home And LandscapeDesign, filed on Nov. 13, 2008, which is a continuation-in-part of U.S.patent application Ser. No. 11/563,549, entitled “Visual Bookmarks ForHome And Landscape Design”, filed on Nov. 27, 2006; U.S. patentapplication Ser. No. 11/563,564, entitled “Converting Web Content IntoTwo-Dimensional CAD Drawings And Three-Dimensional CAD Models”, filed onNov. 27, 2006; U.S. patent application Ser. No. 11/563,569, entitled“Converting Web Content Into Texture Mapping Objects”, filed on Nov. 27,2006; U.S. patent application Ser. No. 11/563,604, entitled “Joining AndDisjoining Individual Rooms In A Floor Plan”, filed on Nov. 27, 2006;and U.S. patent application Ser. No. 11/563,607, entitled “Searching AndMatching Related Objects, Drawings And Models For Home And LandscapeDesign”, filed on Nov. 27, 2006; and which claims priority to U.S.Provisional Patent Application No. 60/992,715, entitled “Systems,Methods, And Computer Program Products For Home And Landscape Design”,filed on Dec. 5, 2007; and U.S. Provisional Patent Application No.61/019,816, entitled “Systems, Methods, And Computer Program ProductsFor Home And Landscape Design”, filed on Jan. 8, 2008. The contents ofU.S. patent application Ser. Nos. 11/563,549; 11/563,564; 11/563,569;11/563,604; 11/563,607; and 12/270,470; and of U.S. Provisional PatentApplication Nos. 60/992,715 and 61/019,816, are hereby incorporated byreference into this application as if set forth herein in full.

TECHNICAL FIELD

This patent application relates generally to systems, methods, andcomputer program products for home and/or landscape design.

BACKGROUND

Designing the layout and products to include in a room can be a timeconsuming process. For example, a person designing a room can selectvarious products to include in the room by collecting pages frommagazines or from the internet. The person designing the room can laterview these pages when selecting a product to purchase. The persondesigning the room may also use a computer-aided design (CAD) program ortool to generate a computer based layout of the room that includes theplacement of various items within the room. If the person designing theroom has questions about the items to include in the room or the layoutof the room, he/she may ask a salesperson at a store about products orask friends for recommendations or opinions about various aspects ofhis/her design.

The typical homeowner will spend months planning a home improvementproject. The result can be an unorganized collection of pages torn outof magazines and catalogs, stacks of books with bookmarked pages,lengthy Internet Explorer “Favorites” lists with poor naming conventionsand sketches of designs that are not to scale.

SUMMARY

This patent application relates generally to systems, methods, andcomputer program products for home and/or landscape design.

In some aspects, a computer-implemented method for use in building athree-dimensional representation of a home design product includesprocessing a graphical image of the home design product. The method alsoincludes associating a three-dimensional base model with the graphicalimage. The method further includes applying a texture to thethree-dimensional base model to build the three-dimensionalrepresentation of the home design product.

Implementations can include one or more of the following.

The computer-implemented method may further include generating a visualbookmark of the home design product. The computer-implemented method mayfurther include scraping information regarding the home design productfrom a website. The information may include the graphical image. Theinformation may include taxonomy and folksonomy data. The informationmay include dimensional information regarding the home design product.

The computer-implemented method may further include receiving thegraphical image. The three-dimensional base model may be built prior tothe graphical image being received. The computer-implemented method mayfurther include applying edge detection to the graphical image to definesurfaces of the graphical image. The computer-implemented method mayalso include cropping images from the surfaces of the graphical image,and applying the cropped images to surfaces of the three-dimensionalbase model using a texture mapping technique to build thethree-dimensional representation. The computer-implemented method mayfurther include selectively ignoring a feature detail of the graphicalimage found while applying edge detection as an extraneous detail. Thecomputer-implemented method may further include presenting thethree-dimensional representation to a user, and requesting that the userverify the three-dimensional representation.

The computer-implemented method may further include defining a productmodel type of the home design product. The computer-implemented methodmay also include retrieving the three-dimensional base model from adatabase according to the product model type.

In the computer-implemented method, processing the graphical image andassociating the three-dimensional base model with the graphical imagemay include building the three-dimensional base model. In thecomputer-implemented method, building the three-dimensional base modelmay include building the three-dimensional base model using at least oneof first information scraped from a website, second information from adatabase of product type attributes, neural networking information,interpretation of the graphical image, or third information provided bya user. Building the three-dimensional base model may also includedetermining an order in which to build portions of the three-dimensionalbase model, where the portions correspond to features of the home designproduct. The order may be determined based on at least one of designrules, first information scraped from a website, second information froma database of product type attributes, neural networking information,interpretation of the graphical image, third information provided by auser, or fourth information regarding methods of building thethree-dimensional model for a product type of the home design product.Building the three-dimensional base model may further include applyingsolid modeling techniques to build the portions of the three-dimensionalbase model according to the order.

In the computer-implemented method, building the three-dimensional basemodel may include determining perspective information for the graphicalimage. Building the three-dimensional base model may further includeapplying edge detection to the graphical image. Building thethree-dimensional base model may further include assigning dimensionalinformation to the graphical image. The dimensional information mayinclude at least one of overall dimensions for the home design product,or dimensions for a feature of the home design product. Building thethree-dimensional base model may further include setting a relativescale for the three-dimensional base model using the dimensionalinformation. Building the three-dimensional base model may furtherinclude determining main object planes based on the graphical image andforming a bounding box from the main object planes. The bounding box mayinclude an outer boundary for the three-dimensional base model. Buildingthe three-dimensional base model may further include applying thedimensional information to the main object planes to define one or moredistances between the main object planes.

In the computer-implemented method, building the three-dimensional basemodel may further include applying edge detection to the graphicalimage. Building the three-dimensional base model may further includedetermining a feature level plane for a feature of the graphical image.Building the three-dimensional base model may further includedetermining main object planes based on the graphical image, forming abounding box from the main object planes, and assigning the featurelevel plane within the bounding box. The bounding box may include anouter boundary for the three-dimensional base model.

In the computer-implemented method, building the three-dimensional basemodel may further include applying edge detection to the graphicalimage. Building the three-dimensional base model may further includedetermining a feature level plane for a feature of the graphical image.Building the three-dimensional base model may further include defining aprofile on the feature level plane, where the profile corresponds to thefeature of the graphical image. Building the three-dimensional basemodel may further include retrieving the profile from a profiledatabase. The profile may be parametric. Building the three-dimensionalbase model may further include adjusting the profile to match a shape ofthe feature of the graphical image. The feature of the graphical imagemay intersect the feature level plane. Building the three-dimensionalbase model may further include drawing the profile to match a shape ofthe feature of the graphical image. The feature of the graphical imagemay intersect the feature level plane. Building the three-dimensionalbase model may further include applying a solid modeling technique tothe profile to define the feature in the three-dimensional base model.Applying the solid modeling technique may include extruding the profileto build a three-dimensional model of the feature. Building thethree-dimensional base model may further include warping a surface ofthe three-dimensional model of the feature. Building thethree-dimensional base model may further include mirroring the profilein the feature level plane.

In the computer-implemented method, building the three-dimensional basemodel may further include applying edge detection to the graphicalimage. Building the three-dimensional base model may further includedetermining a feature level plane for a feature of the graphical image.Building the three-dimensional base model may further include defining aprofile on the feature level plane, where the profile corresponds to thefeature of the graphical image. Building the three-dimensional basemodel may further include determining a second feature level plane forthe feature of the graphical image. Building the three-dimensional basemodel may further include defining a second profile on the secondfeature level plane. The second profile may correspond to the feature ofthe graphical image. Building the three-dimensional base model mayfurther include evaluating the profile and the second profile, selectinga best profile of the profile and the second profile, and applying asolid modeling technique to the best profile to define the feature inthe three-dimensional base model. The computer-implemented method mayfurther include comparing the three-dimensional base model to thegraphical image to determine whether to the three-dimensional base modelis a proper characterization of the home design product, and using thethree-dimensional base model to build the three-dimensionalrepresentation if the three-dimension base model is a propercharacterization of the home design product. The computer-implementedmethod may further include presenting the three-dimensional base modelto a user, and requesting that the user verify the three-dimensionalrepresentation.

In the computer-implemented method, the texture may include a storedtexture. The texture may include a cropped image from the graphicalimage. In the computer-implemented method, applying a texture mayinclude applying the texture using a texture mapping technique. Thetexture mapping technique may includes at least one of tiling thetexture, warping the texture, or stretching the texture. Thecomputer-implemented method may further include applying a texturaldetail to the three-dimensional base model to build thethree-dimensional representation. The textural detail may include a bumpmap.

The computer-implemented method may further include presenting thethree-dimensional representation to a user, and requesting that the userverify the three-dimensional representation. The computer-implementedmethod may further include providing an interface to a user to allow auser to edit at least one of the three-dimensional representation or thethree-dimensional model. The computer-implemented method may furtherinclude building a two-dimensional drawing of the home design productusing the three-dimensional representation.

In some aspects, a computer-implemented method for use in generating ahome design layout includes modifying a structural design layout withone or more structural elements. The method also includes inserting oneor more home design product elements into the modified structural designlayout in conformance with the one or more structural elements to builda set of home design product layouts. The method further includespresenting the set of home design product layouts to a user. The methodalso includes presenting sets of home design product images to the user.The method further includes, responsive to a request, converting a homedesign product layout of the set of home design product layouts to athree-dimensional layout. The three-dimensional layout incorporatesthree-dimensional models associated with corresponding home designproduct images from a set of the sets of home design product images.

Implementations can include one or more of the following.

The computer-implemented method can further include presentingadditional home design product images to the user. The additional homedesign product images can be associated with corresponding additionalthree-dimensional models. Presenting the additional home design productimages to the user can include selecting the additional home designproduct images for presentation based on the three-dimensional modelsincorporated into the three-dimensional layout. Presenting theadditional home design product images can include selecting theadditional home design product images for presentation based on one ormore of the following: design rules, sponsorship by advertisers, inputpreviously provided by the user, design choices previously made by theuser, or neural network analysis of design choices of other users.Presenting the additional home design product images can includebuilding the additional home design product images for presentationbased on one or more of the following: design rules, sponsorship byadvertisers, input previously provided by the user, design choicespreviously made by the user, or neural network analysis of designchoices of other users. Presenting the additional home design productimages can include presenting the additional home design product imagesin order of one or more of the following: most often used by otherusers, most recently used by other users, level of sponsorship byadvertisers, whether used by design professionals, input previouslyprovided by the user, or design choices previously made by the user.Presenting the additional home design product images can includeincorporating one or more of the additional three-dimensional modelsinto the three-dimensional layout. Presenting the additional home designproduct images can include building a new set of home design productimages corresponding to the three-dimensional models and the additionalthree-dimensional models.

The computer-implemented method can further include, responsive to arequest, adding a new three-dimensional model or removing one or more ofthe three-dimensional models incorporated into the three-dimensionallayout to build a new set of three-dimensional models incorporated intothe three-dimensional layout, and building a new set of home designproduct images. Each home design product image of the new set of homedesign product images can be associated with a correspondingthree-dimensional model of the new set of three-dimensional models. Themethod can further include prompting the user for information regardingthe new set of home design product images, and associating theinformation with and storing the new set of home design product images.The information can include least one of a design style; the user'soccupation; or a description.

In the computer-implemented method, the one or more structural elementscan include one or more two-dimensional drawings. The one or morestructural elements can include at least one of a room, a window, adoor, a door opening, a closet, a set of stairs, an additional floorlevel, or a fireplace. The structural design layout can be modifiedautomatically. The structural design layout can be modified responsiveto a first request. The one or more home design product elements caninclude one or more two-dimensional drawings. The one or more homedesign product elements can include at least one of a home designproduct, a closet, a set of kitchen cabinets, a table, or a bed. Eachhome design product image of the sets of home design product images canbe associated with a corresponding two-dimensional drawing andthree-dimensional model of a home design product shown in the homedesign product image.

The computer-implemented method can include presenting sets ofstructural elements to the user. The one or more structural elements canbe drawn from the set of home design product elements. Thecomputer-implemented method can further include applying exteriorstructural elements to the modified structural design layout. The methodcan also include presenting sets of home design product elements to theuser. The one or more home design product elements can be drawn from theset of home design product elements.

In the computer-implemented method, inserting the one or more homedesign product elements into the modified structural design layout, andpresenting the set of home design product layouts, can include selectingthe set of home design product layouts for presentation based on one ormore of the following: design rules, sponsorship by advertisers, inputpreviously provided by the user, design choices previously made by theuser, or neural network analysis of design choices of other users.Inserting the one or more home design product elements into the modifiedstructural design layout, and presenting the set of home design productlayouts, can also include presenting home design product layouts withinthe set of home design product layouts in order of one or more of thefollowing: most often used by other users, most recently used by otherusers, level of sponsorship by advertisers, whether used by designprofessionals, input previously provided by the user, or design choicespreviously made by the user.

In the computer-implemented method, inserting the one or more homedesign product elements into the modified structural design layout, andpresenting the set of home design product layouts, can include buildingthe set of home design product layouts for presentation based on one ormore of the following: design rules, sponsorship by advertisers, inputpreviously provided by the user, design choices previously made by theuser, or neural network analysis of design choices of other users.Inserting the one or more home design product elements into the modifiedstructural design layout, and presenting the set of home design productlayouts, can also include presenting home design product layouts withinthe set of home design product layouts in order of one or more of one ormore of the following: most often used by other users, most recentlyused by other users, level of sponsorship by advertisers, whether usedby design professionals, input previously provided by the user, ordesign choices previously made by the user.

In the computer-implemented method, inserting the one or more homedesign product elements into the modified structural design layout, andpresenting the set of home design product layouts, can include buildingone or more home design product layouts of the set of home designproduct layouts for presentation, and can also include moving one ormore redundant home design product layouts out of the set of home designproduct layouts before presenting the set of home design product layoutsto the user.

In the computer-implemented method, presenting the sets of home designproduct images can include selecting the sets of home design productimages for presentation based on the one or more home design productelements inserted into the modified structural design layout. Presentingthe sets of home design product images can include building the sets ofhome design product images for presentation based on the one or morehome design product elements inserted into the modified structuraldesign layout. Presenting the sets of home design product images caninclude selecting the sets of home design product images forpresentation based on one or more of the following: design rules,sponsorship by advertisers, input previously provided by the user,design choices previously made by the user, or neural network analysisof design choices of other users. Presenting the sets of home designproduct images can include building the sets of home design productimages for presentation based on one or more of the following: designrules, sponsorship by advertisers, input previously provided by theuser, design choices previously made by the user, or neural networkanalysis of design choices of other users. Presenting the sets of homedesign product images can include presenting the sets of home designproduct images in order of one or more of the following: most often usedby other users, most recently used by other users, level of sponsorshipby advertisers, whether used by design professionals, input previouslyprovided by the user, or design choices previously made by the user.

In the computer-implemented method, converting the home design productlayout of the set of home design product layouts can include inserting aset of the sets of home design product images into the home designproduct layout.

In the computer-implemented method, the sets of home design productimages can include surface texture images. The three-dimensional modelscan include surface texture images mapped to a surface in thethree-dimensional layout.

In some aspects, a computer program product is tangibly embodied in oneor more information carriers and includes instructions that areexecutable by one or more processing devices to modify a structuraldesign layout with one or more structural elements; insert one or morehome design product elements into the modified structural design layoutin conformance with the one or more structural elements to build a setof home design product layouts; present the set of home design productlayouts to a user; present sets of home design product images to theuser; and, responsive to a request, convert a home design product layoutof the set of home design product layouts to a three-dimensional layout.The three-dimensional layout incorporates three-dimensional modelsassociated with corresponding home design product images from a set ofthe sets of home design product images.

Implementations can include one or more of the following.

The computer program product can also include instructions that areexecutable by the one or more processing devices to present additionalhome design product images to the user, the additional home designproduct images associated with corresponding additionalthree-dimensional models. Presenting the additional home design productimages can include selecting the additional home design product imagesfor presentation based on the three-dimensional models incorporated intothe three-dimensional layout. Presenting the additional home designproduct images can include selecting the additional home design productimages for presentation based on one or more of the following: designrules, sponsorship by advertisers, input previously provided by theuser, design choices previously made by the user, or neural networkanalysis of design choices of other users. Presenting the additionalhome design product images can include building the additional homedesign product images for presentation based on one or more of thefollowing: design rules, sponsorship by advertisers, input previouslyprovided by the user, design choices previously made by the user, orneural network analysis of design choices of other users. Presenting theadditional home design product images can include presenting theadditional home design product images in order of one or more of thefollowing: most often used by other users, most recently used by otherusers, level of sponsorship by advertisers, whether used by designprofessionals, input previously provided by the user, or design choicespreviously made by the user. Presenting the additional home designproduct images can include incorporating one or more of the additionalthree-dimensional models into the three-dimensional layout. Presentingthe additional home design product images can include building a new setof home design product images corresponding to the three-dimensionalmodels and the additional three-dimensional models.

The computer program product can also include instructions that areexecutable by the one or more processing devices to, responsive to arequest, add a new three-dimensional model or removing one or more ofthe three-dimensional models incorporated into the three-dimensionallayout to build a new set of three-dimensional models incorporated intothe three-dimensional layout; and build a new set of home design productimages. Each home design product image of the new set of home designproduct images can be associated with a corresponding three-dimensionalmodel of the new set of three-dimensional models. The computer programproduct can also include instructions that are executable by the one ormore processing devices to prompt the user for information regarding thenew set of home design product images, and associate the informationwith and storing the new set of home design product images. Theinformation can include at least one of a design style; the user'soccupation; or a description.

In the computer program product, the one or more structural elements caninclude one or more two-dimensional drawings. The one or more structuralelements can include at least one of a room, a w, a window, a door, adoor opening, a closet, a set of stairs, an additional floor level, or afireplace. The structural design layout can be modified automatically.The structural design layout can be modified responsive to a firstrequest. The one or more home design product elements can include one ormore two-dimensional drawings. The one or more home design productelements comprise at least one of a home design product, a closet, a setof kitchen cabinets, a table, or a bed. Each home design product imageof the sets of home design product images can be associated with acorresponding two-dimensional drawing and three-dimensional model of ahome design product shown in the home design product image.

The computer program product can also include instructions that areexecutable by the one or more processing devices to present sets ofstructural elements to the user. The one or more structural elements canbe drawn from the set of home design product elements. The computerprogram product can also include instructions that are executable by theone or more processing devices to apply exterior structural elements tothe modified structural design layout. The computer program product canalso include instructions that are executable by the one or moreprocessing devices to present sets of home design product elements tothe user. The one or more home design product elements can be drawn fromthe set of home design product elements.

In the computer program product, inserting the one or more home designproduct elements into the modified structural design layout, andpresenting the set of home design product layouts, can include selectingthe set of home design product layouts for presentation based on one ormore of the following: design rules, sponsorship by advertisers, inputpreviously provided by the user, design choices previously made by theuser, or neural network analysis of design choices of other users; andpresenting home design product layouts within the set of home designproduct layouts in order of one or more of the following: most oftenused by other users, most recently used by other users, level ofsponsorship by advertisers, whether used by design professionals, inputpreviously provided by the user, or design choices previously made bythe user.

In the computer program product, inserting the one or more home designproduct elements into the modified structural design layout, andpresenting the set of home design product layouts, can include buildingthe set of home design product layouts for presentation based on one ormore of the following: design rules, sponsorship by advertisers, inputpreviously provided by the user, design choices previously made by theuser, or neural network analysis of design choices of other users; andpresenting home design product layouts within the set of home designproduct layouts in order of one or more of the following: most oftenused by other users, most recently used by other users, level ofsponsorship by advertisers, whether used by design professionals, inputpreviously provided by the user, or design choices previously made bythe user.

In the computer program product, inserting the one or more home designproduct elements into the modified structural design layout, andpresenting the set of home design product layouts, can include buildingone or more home design product layouts of the set of home designproduct layouts for presentation; and moving one or more redundant homedesign product layouts out of the set of home design product layoutsbefore presenting the set of home design product layouts to the user.

In the computer program product, presenting the sets of home designproduct images can include selecting the sets of home design productimages for presentation based on the one or more home design productelements inserted into the modified structural design layout. Presentingthe sets of home design product images can include building the sets ofhome design product images for presentation based on the one or morehome design product elements inserted into the modified structuraldesign layout. Presenting the sets of home design product images caninclude selecting the sets of home design product images forpresentation based on one or more of the following: design rules,sponsorship by advertisers, input previously provided by the user,design choices previously made by the user, or neural network analysisof design choices of other users. Presenting the sets of home designproduct images can include building the sets of home design productimages for presentation based on one or more of the following: designrules, sponsorship by advertisers, input previously provided by theuser, design choices previously made by the user, or neural networkanalysis of design choices of other users. Presenting the sets of homedesign product images can include presenting the sets of home designproduct images in order of one or more of the following: most often usedby other users, most recently used by other users, level of sponsorshipby advertisers, whether used by design professionals, input previouslyprovided by the user, or design choices previously made by the user.

In the computer program product, converting the home design productlayout of the set of home design product layouts can include inserting aset of the sets of home design product images into the home designproduct layout.

In the computer program product, the sets of home design product imagescan include surface texture images; and the three-dimensional models caninclude surface texture images mapped to a surface in thethree-dimensional layout.

In some aspects, a system includes an apparatus configured to generate ahome design layout. The apparatus includes memory configured to storeinstructions for execution, and one or more processing devicesconfigured to execute the instructions. The instructions are for causingthe one or more processing devices to modify a structural design layoutwith one or more structural elements; insert one or more home designproduct elements into the modified structural design layout inconformance with the one or more structural elements to build a set ofhome design product layouts; present the set of home design productlayouts to a user; present sets of home design product images to theuser; and, responsive to a request, convert a home design product layoutof the set of home design product layouts to a three-dimensional layout.The three-dimensional layout incorporates three-dimensional modelsassociated with corresponding home design product images from a set ofthe sets of home design product images.

Implementations can include one or more of the following.

The system can also include instructions that are executable by the oneor more processing devices to present additional home design productimages to the user, the additional home design product images associatedwith corresponding additional three-dimensional models. Presenting theadditional home design product images can include selecting theadditional home design product images for presentation based on thethree-dimensional models incorporated into the three-dimensional layout.Presenting the additional home design product images can includeselecting the additional home design product images for presentationbased on one or more of the following: design rules, sponsorship byadvertisers, input previously provided by the user, design choicespreviously made by the user, or neural network analysis of designchoices of other users. Presenting the additional home design productimages can include building the additional home design product imagesfor presentation based on one or more of the following: design rules,sponsorship by advertisers, input previously provided by the user,design choices previously made by the user, or neural network analysisof design choices of other users. Presenting the additional home designproduct images can include presenting the additional home design productimages in order of one or more of one or more of the following: mostoften used by other users, most recently used by other users, level ofsponsorship by advertisers, whether used by design professionals, inputpreviously provided by the user, or design choices previously made bythe user. Presenting the additional home design product images caninclude incorporating on or more of the additional three-dimensionalmodels into the three-dimensional layout. Presenting the additional homedesign product images can include building a new set of home designproduct images corresponding to the three-dimensional models and theadditional three-dimensional models.

The system can also include instructions that are executable by the oneor more processing devices to, responsive to a request, add a newthree-dimensional model or removing one or more of the three-dimensionalmodels incorporated into the three-dimensional layout to build a new setof three-dimensional models incorporated into the three-dimensionallayout; and build a new set of home design product images. Each homedesign product image of the new set of home design product images can beassociated with a corresponding three-dimensional model of the new setof three-dimensional models. The system can also include instructionsthat are executable by the one or more processing devices to prompt theuser for information regarding the new set of home design productimages, and associate the information with and storing the new set ofhome design product images. The information can include at least one ofa design style; the user's occupation; or a description.

In the system, the one or more structural elements can include one ormore two-dimensional drawings. The one or more structural elements caninclude at least one of a room, a window, a door, a door opening, acloset, a set of stairs, an additional floor level, or a fireplace. Thestructural design layout can be modified automatically. The structuraldesign layout can be modified responsive to a first request. The one ormore home design product elements can include one or moretwo-dimensional drawings. The one or more home design product elementscomprise at least one of a home design product, a closet, a set ofkitchen cabinets, a table, or a bed. Each home design product image ofthe sets of home design product images can be associated with acorresponding two-dimensional drawing and three-dimensional model of ahome design product shown in the home design product image.

The system can also include instructions that are executable by the oneor more processing devices to present sets of structural elements to theuser. The one or more structural elements can be drawn from the set ofhome design product elements. The system can also include instructionsthat are executable by the one or more processing devices to applyexterior structural elements to the modified structural design layout.The system can also include instructions that are executable by the oneor more processing devices to present sets of home design productelements to the user. The one or more home design product elements canbe drawn from the set of home design product elements.

In the system, inserting the one or more home design product elementsinto the modified structural design layout, and presenting the set ofhome design product layouts, can include selecting the set of homedesign product layouts for presentation based on one or more of thefollowing: design rules, sponsorship by advertisers, input previouslyprovided by the user, design choices previously made by the user, orneural network analysis of design choices of other users; and presentinghome design product layouts within the set of home design productlayouts in order of one or more of the following: most often used byother users, most recently used by other users, level of sponsorship byadvertisers, whether used by design professionals, input previouslyprovided by the user, or design choices previously made by the user.

In the system, inserting the one or more home design product elementsinto the modified structural design layout, and presenting the set ofhome design product layouts, can include building the set of home designproduct layouts for presentation based on one or more of the following:design rules, sponsorship by advertisers, input previously provided bythe user, design choices previously made by the user, or neural networkanalysis of design choices of other users; and presenting home designproduct layouts within the set of home design product layouts in orderof one or more of the following: most often used by other users, mostrecently used by other users, level of sponsorship by advertisers,whether used by design professionals, input previously provided by theuser, or design choices previously made by the user.

In the system, inserting the one or more home design product elementsinto the modified structural design layout, and presenting the set ofhome design product layouts, can include building one or more homedesign product layouts of the set of home design product layouts forpresentation; and moving one or more redundant home design productlayouts out of the set of home design product layouts before presentingthe set of home design product layouts to the user.

In the system, presenting the sets of home design product images caninclude selecting the sets of home design product images forpresentation based on the one or more home design product elementsinserted into the modified structural design layout, Presenting the setsof home design product images can include building the sets of homedesign product images for presentation based on the one or more homedesign product elements inserted into the modified structural designlayout. Presenting the sets of home design product images can includeselecting the sets of home design product images for presentation basedon one or more of the following: design rules, sponsorship byadvertisers, input previously provided by the user, design choicespreviously made by the user, or neural network analysis of designchoices of other users. Presenting the sets of home design productimages can images can include building the sets of home design productimages for presentation based on one or more of the following: designrules, sponsorship by advertisers, input previously provided by theuser, design choices previously made by the user, or neural networkanalysis of design choices of other users. Presenting the sets of homedesign product images can include presenting the sets of home designproduct images in order of one or more of the following: most often usedby other users, most recently used by other users, level of sponsorshipby advertisers, whether used by design professionals, input previouslyprovided by the user, or design choices previously made by the user.

In the system, converting the home design product layout of the set ofhome design product layouts can include inserting a set of the sets ofhome design product images into the home design product layout.

In the system, the sets of home design product images can includesurface texture images; and the three-dimensional models can includesurface texture images mapped to a surface in the three-dimensionallayout.

In some aspects, a computer-implemented method for providing a user witha home design interface includes, in a two-dimensional structural designmode, presenting a user with one or more structural elements to modify atwo-dimensional structural design layout. The method also includes, in athree-dimensional structural design mode, presenting a user with one ormore exterior structural elements to modify a three-dimensionalstructural design layout. The three-dimensional structural design layoutis based on the two-dimensional structural design layout. The methodalso includes, in a two-dimensional interior design mode, presenting auser with one or more product layouts; and presenting a user with one ormore sets of product images to apply to a product layout of the one ormore product layouts. The one or more product layouts are based on leastone of the two-dimensional structural design layout or thethree-dimensional structural design layout. The method also includes, ina three-dimensional interior design mode, presenting a user with one ormore sets of complementary product images to apply to athree-dimensional design layout. The three-dimensional design layout isbased on the product layout and incorporates three-dimensional modelsassociated with corresponding product images from the one or more setsof product images. The method also includes allowing the user to cycleback and forth between the modes.

In some aspects, a computer program product is tangibly embodied in oneor more information carriers and includes instructions that areexecutable by one or more processing devices to, in a two-dimensionalstructural design mode, present a user with one or more structuralelements to modify to modify a two-dimensional structural design layout.The instructions are also executable to, in a three-dimensionalstructural design mode, present a user with one or more exteriorstructural elements to modify a three-dimensional structural designlayout. The three-dimensional structural design layout are based on thetwo-dimensional structural design layout. The instructions are alsoexecutable to, in a two-dimensional interior design mode, present a userwith one or more product layouts; and present a user with one or moresets of product images to apply to a product layout of the one or moreproduct layouts. The one or more product layouts are based on least oneof the two-dimensional structural design layout or the three-dimensionalstructural design layout. The instructions are also executable to, in athree-dimensional interior design mode, present a user with one or moresets of complementary product images to apply to a three-dimensionaldesign layout. The three-dimensional design layout is based on theproduct layout and incorporates three-dimensional models associated withcorresponding product images from the one or more sets of productimages. The instructions are also executable to allow the user to cycleback and forth between the modes.

In, some aspects, a system includes an apparatus configured to provide auser with a home design interface. The apparatus includes memoryconfigured to store instructions for execution, and one or moreprocessing devices configured to execute the instructions. Theinstructions are for causing the one or more processing devices to, in atwo-dimensional structural design mode, present a user with one or morestructural elements to modify a two-dimensional structural designlayout. The instructions are also for causing the one or more processingdevices to, in a three-dimensional structural design mode, present auser with one or more exterior structural elements to modify athree-dimensional structural design layout. The three-dimensionalstructural design layout are based on the two-dimensional structuraldesign layout. The instructions are also for causing the one or moreprocessing devices to, in a two-dimensional interior design mode,present a user with one or more product layouts; and present a user withone or more sets of product images to apply to a product layout of theone or more product layouts. The one or more product layouts are basedon least one of the two-dimensional structural design layout or thethree-dimensional structural design layout. The instructions are alsofor causing the one or more processing devices to, in athree-dimensional interior design mode, present a user with one or moresets of complementary product images to apply to a three-dimensionaldesign layout. The three-dimensional design layout is based on theproduct layout and incorporates three-dimensional models associated withcorresponding product images from the one or more sets of productimages. The instructions are also for causing the one or more processingdevices to allow the user to cycle back and forth between the modes.

In some aspects, a computer-implemented method for building a productlayout includes receiving a structural design layout. The structuraldesign layout includes one or more structural elements. The method alsoincludes building one or more product layouts from the structural designlayout by inserting a first product element into the structural designlayout in conformance with the one or more structural elements accordingto a first design rule; and inserting a second product element into thestructural design layout in conformance with the one or more structuralelements and the first product element and according to a second designrule.

Implementations can include one or more of the following.

In the computer-implemented method, the second product element caninclude kitchen cabinets.

In some aspects, a computer program product is tangibly embodied in oneor more information carriers and includes instructions that areexecutable by one or more processing devices to receive a structuraldesign layout. The structural design layout includes one or morestructural elements. The instructions are also executable to build oneor more product layouts from the structural design layout by inserting afirst product element into the structural design layout in conformancewith the one or more structural elements according to a first designrule; and inserting a second product element into the structural designlayout in conformance with the one or more structural elements and thefirst product element and according to a second design rule.

Implementations can include one or more of the following.

In the computer program product, the second product element can includekitchen cabinets.

In some aspects, a system includes an apparatus configured to build aproduct layout. The apparatus includes memory configured to storeinstructions for execution, and one or more processing devicesconfigured to execute the instructions. The instructions are for causingthe one or more processing devices to receive a structural designlayout. The structural design layout includes one or more structuralelements. The instructions are also for causing the one or moreprocessing devices to build one or more product layouts from thestructural design layout by inserting a first product element into thestructural design layout in conformance with the one or more structuralelements according to a first design rule; and inserting a secondproduct element into the structural design layout in conformance withthe one or more structural elements and the first product element andaccording to a second design rule.

Implementations can include one or more of the following.

In the system, the second product element can include kitchen cabinets.

In some aspects, a computer-implemented method for suggesting homedesign products to a user includes presenting a user with one or moresets of complementary product images to apply to a three-dimensionaldesign layout. The three-dimensional design layout is based on a productlayout and incorporates three-dimensional models associated withcorresponding product images from one or more sets of product images.

In some aspects, a computer-implemented method for presenting homedesign product sets to a user includes receiving a first product imageof a first product from a user. The method also includes presenting setsof product images to a user. The sets of product images each include asecond product image related to the first product image. The secondproduct image is an image of a second product that shares one or morecharacteristics with the first product. Each product image of the setsof product images is associated with a corresponding two-dimensionaldrawing and three-dimensional model of a product shown in the productimage. The method also includes generating a three-dimensional designlayout using a set of the set of product images.

More particularly, in some aspects, this application relates to a systemand method for generating visual bookmarks for home improvement productsby scraping, extracting, and categorizing information.

In some aspects, a computer-implemented method for use in generating ahome design layout includes receiving a request to generate a visualbookmark for a home design product based on a selection of the homedesign product from a website and extracting information about the homedesign product from the website. The method also includes categorizingthe home design product based on the information and generating thevisual bookmark for the home design product, the visual bookmarkcomprising an image of the home design product and information relatedto the home design product.

Implementations can include one or more of the following.

Extracting information about the home design product from the websitecan include searching the website for words used to identify aparticular type of information about the home design product. The wordscan be one or more words in a folksonomy based system. Generating thevisual bookmark can include extracting the image of the home designproduct from the website and resizing the image of the home designproduct for the visual bookmark. The information related to the productcan include one or more of a style of the home design product, a colorof the home design product, and dimensional information about the homedesign product. Categorizing the home design product can includepre-selecting a product category based on the extracted information,pre-selecting a product type based on the extracted information,presenting the pre-selected product category, presenting thepre-selected product type and receiving feedback on the pre-selectedproduct category and product type. Categorizing the home design productcan include identifying a taxonomy based category for the home designproduct based on information associated with the home design product andthe received feedback on the pre-selected product category and producttype. Receiving feedback on the pre-selected product category andproduct type from the user can include receiving confirmation that thepre-selected category and product type are correct. Receiving feedbackcan include receiving a selection of at least one of a product categoryand a product type that is different from the pre-selected productcategory and pre-selected product type and storing the receivedselection.

In some aspects, a computer program product is tangibly embodied in aninformation carrier for executing instructions on a processor. Thecomputer program product is operable to cause a machine to receive arequest to generate a visual bookmark for a home design product based ona selection of the home design product from a website, extractinformation about the home design product from the website, categorizethe home design product based on the information, and generate thevisual bookmark for the home design product. The visual bookmarkincludes an image of the home design product and information related tothe home design product.

Implementations can include one or more of the following.

The instructions to extract information about the home design productfrom the website can include instructions to cause a machine to searchthe website for words used to identify a particular type of informationabout the home design product. The instructions to generate the visualbookmark can include instructions to cause a machine to extract theimage of the home design product from the website and resize the imageof the home design product for the visual bookmark. The instructions tocategorize the home design product can include instructions to cause amachine to pre-select a product category based on the extractedinformation, pre-select a product type based on the extractedinformation, present the pre-selected product category, present thepre-selected product type, and receive feedback on the pre-selectedproduct category and product type. The instructions to categorize thehome design product can include instructions to cause a machine toidentify a taxonomy based category for the home design product based oninformation associated with the home design product and the receivedfeedback on the pre-selected product category and product type.

In some aspects a system for use in generating a home design layout isconfigured to receive a request to generate a visual bookmark for a homedesign product based on a selection of the home design product from awebsite, extract information about the home design product from thewebsite, categorize the home design product based on the information,and generate the visual bookmark for the home design product, the visualbookmark comprising an image of the home design product and informationrelated to the home design product.

Implementations can include one or more of the following.

The system can be further configured to search the website for wordsused to identify a particular type of information about the home designproduct. The system can be further configured to extract the image ofthe home design product from the website and resize the image of thehome design product for the visual bookmark. The system can be furtherconfigured to pre-select a product category based on the extractedinformation, pre-select a product type based on the extractedinformation, present the pre-selected product category, present thepre-selected product type, and receive feedback on the pre-selectedproduct category and product type. The system can be further configuredto identify a taxonomy based category for the home design product basedon information associated with the home design product and the receivedfeedback on the pre-selected product category and product type.

In other aspects, this application relates to scraping and extractinginformation from websites and converting this information into 2D CAD(computer-aided design) drawings and 3D CAD models.

In some aspects, a computer-implemented method for use in generating ahome design layout includes extracting dimensional information about ahome design product from a website, extracting properties about a homedesign product from the website, extracting categorization informationabout the home design product from the website, and identifying, basedon the categorization information and the dimensional information, oneor more pre-existing computer-aided design (CAD) models stored in adatabase of CAD models.

Implementations can include one or more of the following.

The method can also include presenting the one or more CAD models to auser; receiving a selection of a particular one of the CAD modelsselected from the one or more identified CAD models, and associating theselected particular one of the CAD models can include determining alikelihood of a match between each of the one or more identified CADmodels and the home design product based on the extracted categorizationand dimensional information and historical information stored aboutpreviously selected CAD models for the home design product andpresenting the one or more CAD models in an order, the order being basedon the likelihood of a match between the CAD model and the home designproduct. The method can also include receiving a request to generate aCAD model for a particular home design product based on a user selectionof the home design product from a website. The CAD model can be atwo-dimensional CAD model. The CAD model can be a three-dimensional CADmodel. The method can also include generating a visual bookmarkassociated with the home design product and associating the visualbookmark with the selected particular one of the CAD models. The methodcan also include inserting the particular one of the CAD models into ahome design layout based on a user selection of the visual bookmarkassociated with the particular one of the CAD models.

In some aspects, a computer program product is tangibly embodied in aninformation career for executing instructions on a processor. Thecomputer program product is operable to cause a machine to receive arequest to extract dimensional information about a home design productfrom a website, extract properties about a home design product from thewebsite, extract categorization information about the home designproduct from the website, and identify, based on the categorizationinformation and the dimensional information, one or more pre-existingcomputer-aided design (CAD) models stored in a database of CAD models.

Implementations can include one or more of the following.

The computer program product can also include instructions to presentthe one or more CAD models to a user, receive a selection of aparticular one of the CAD models selected from the one or moreidentified CAD models, and associate the selected particular one of theCAD models with the home design product. The instructions to cause amachine to identify the one or more identified CAD models can includeinstructions to cause a machine to determine a likelihood of a matchbetween each of the one or more identified CAD models and the homedesign product based on the extracted categorization and dimensionalinformation and historical information stored about previously selectedCAD models for the home design product and present the one or more CADmodels in an order, the order being based on the likelihood of a matchbetween the CAD model and the home design product. The computer programproduct can also include instruction to receive a request to generate aCAD model for a particular home design product based on a user selectionof the home design product from a website. The computer program productcan also include instruction to generate a visual bookmark associatedwith the home design product and associate the visual bookmark with theselected particular one of the CAD models. The computer program productcan also include instruction to insert the particular one of the CADmodels into a home design layout based on a user selection of the visualbookmark associated with the particular one of the CAD models.

In some aspects a system for use in generating a home design layout isconfigured to extract dimensional information about a home designproduct from a website, extract properties about a home design productfrom the website, extract categorization information about the homedesign product from the website, and identify based on thecategorization information and the dimensional information, one or morepre-existing computer-aided design (CAD) models stored in a database ofCAD models.

Implementations can include one or more of the following.

The system can be further configured to present the one or more CADmodels to a user, receive a selection of a particular one of the CADmodels selected from the one or more identified CAD models, andassociate the selected particular one of the CAD models with the homedesign product. The system can be further configured to determine alikelihood of a match between each of the one or more identified CADmodels and the home design product based on the extractedcategorization, and dimensional information and historical informationstored about previously selected CAD models for the home design productand present the one or more CAD models in an order, the order beingbased on the likelihood of a match between the CAD model and the homedesign product. The system can be further configured to receive arequest to generate a CAD model for a particular home design productbased on a user selection of the home design product from a website. Thesystem can be further configured to generate a visual bookmarkassociated with the home design product and associate the visualbookmark with the selected particular one of the CAD models. The systemcan be further configured to insert the particular one of the CAD modelsinto a home design layout based on a user selection of the visualbookmark associated with the particular one of the CAD models.

In other aspects, a computer-implemented method for use in generating ahome design layout includes extracting sizing information about a homedesign product, the home design product having a repeating texture thatis displayed in an image on a webpage, receiving information from auser, the information comprising information about an amount ofrepetition of the repeating texture displayed in the image, andreplicating the image on a surface in a three-dimensional model of aroom based on the extracted information and the received information.

Implementations can include one or more of the following.

The home design product can be tile flooring. The method can alsoinclude extracting sizing information about the tile flooring comprisesextracting a length and a width of one tile and receiving informationfrom the user comprises receiving a number of tiles displayed in theimage. The home design product can be hardwood flooring. The method canalso include extracting sizing information about the hardwood flooringcomprises extracting a width of one piece of hardwood flooring andreceiving information from the user comprises receiving a number ofpieces of hardwood flooring displayed in the image. The home designproduct can be carpet flooring. The method can also include receivinginformation from the user comprises receiving an indication of anapproximate size of a piece of carpet displayed in the image. The methodcan also include generating a visual bookmark associated with the homedesign product. Extracting information about the home design product caninclude searching the website for words used to identify a particulartype of information about the home design product.

In some aspects, a computer program product is tangibly embodied in aninformation carrier for executing instructions on a processor. Thecomputer program product is operable to cause a machine to receive arequest to extract sizing information about a home design product, thehome design product having a repeating texture that is displayed in animage on a webpage, receive information from a user, the informationcomprising information about an amount of repetition of the repeatingtexture displayed in the image, and replicate the image on a surface ina three-dimensional model of a room based on the extracted informationand the received information.

Implementations can include one or more of the following.

The computer program product can also include instruction to generate avisual bookmark associated with the home design product. The computerprogram product can also include instruction to search the website forwords used to identify a particular type of information about the homedesign product.

In some aspects a system for use in generating a home design layout isconfigured to extract sizing information about a home design product,the home design product having a repeating texture that is displayed inan image on a webpage, receive information from a user, the informationcomprising information about an amount of repetition of the repeatingtexture displayed in the image, and replicate the image on a surface ina three-dimensional model of a room based on the extracted informationand the received information.

Implementations can include one or more of the following.

The system can be further configured to generate a visual bookmarkassociated with the home design product. The system can be furtherconfigured to search the website for words used to identify a particulartype of information about the home design product.

In other aspects, this application relates to a system and method forseparating a floor plan in a database into separate rooms. In someadditional aspects, this application relates to joining adjacent wallsof two distinct rooms.

In some aspects, a computer-implemented method for use in generating ahome design layout includes receiving a floor plan from a user thatincludes multiple adjoining rooms, dividing, using a computer system,the floor plan that includes the multiple adjoining rooms intoindividual non-adjoined rooms, and storing information about the layoutof the individual non-adjoined rooms in a database.

Implementations can include one or more of the following.

Dividing the floor plan into individual non-adjoined rooms can includedetermining one or more walls from the floor plan to include in theinformation about a particular room. The method can also includedetermining if any home design products are associated with a particularroom in the floor plan and storing information about the home designproducts associated with the particular room with the information aboutthe layout of the particular room in the database. The method can alsoinclude receiving a user selection of a particular room in the floorplan and presenting, on a user interface, a computer-aided design (CAD)model of the particular room without presenting the CAD models foradjoining rooms. The method can also include searching the database forother rooms that have a similar layout to one of the individualnon-adjoined rooms. The method can also include updating informationabout one of the individual non-adjoined rooms.

In some aspects, a computer-implemented method for use in generating ahome design layout includes receiving a request, the request comprisinga request to join a first room and a second room, the first room havinga wall and the second room having a wall, assigning a priority to one ofthe first and second rooms, the priority indicating a preference formerging the walls of the first and second rooms, and merging the wall ofthe first room and the wall of the second room based on the assignedpriority.

Implementations can include one or more of the following.

Merging the wall of the first room and the wall of the second room basedon the assigned priority can include determining a placement of a doorbased on the location of a door in the room assigned the priority.Merging the wall of the first room and the wall of the second room basedon the assigned priority can include joining first and second roomsusing the wall from the room assigned the priority. Assigning thepriority can include receiving a user input of a preference of one ofthe first and second rooms. Assigning the priority can includeautomatically selecting one of the first and second rooms based on oneor more features of the wall of the first room and the wall of thesecond room.

In some aspects, a computer program product is tangibly embodied in aninformation carrier for executing instructions on a processor. Thecomputer program product is operable to cause a machine to receive arequest to receive a floor plan from a user that includes multipleadjoining rooms, divide the floor plan that includes the multipleadjoining rooms into individual non-adjoined rooms, and storeinformation about the layout of the individual non-adjoined rooms in adatabase.

Implementations can include one or more of the following.

The instructions to divide the floor plan into individual non-adjoinedrooms can include instructions to cause a machine to determine one ormore walls from the floor plan to include in the information about aparticular room. The computer program product can also includeinstruction to determine if any home design products are associated witha particular room in the floor plan and store information about the homedesign products associated with the particular room with the informationabout the layout of the particular room in the database. The computerprogram product can also include instruction to receive a user selectionof a particular room in the floor plan and present, on a user interface,a computer-aided design (CAD) model of the particular room withoutpresenting the CAD models for adjoining rooms. The computer programproduct can also include instruction to search the database for otherrooms that have a similar layout to one of the individual non-adjoinedrooms.

In some aspects, a computer program product is tangibly embodied in aninformation carrier for executing instructions on a processor. Thecomputer program product is operable to cause a machine to receive arequest, the request comprising a request to join a first room and asecond room, the first room having a wall and the second room having awall, assign a priority to one of the first and second rooms, thepriority indicating a preference for merging the walls of the first andsecond rooms, and merge the wall of the first room and the wall of thesecond room based on the assigned priority.

Implementations can include one or more of the following.

The instructions to cause a machine to merge the wall of the first roomand the wall of the second room based on the assigned priority caninclude instructions to cause a machine to determine a placement of adoor based on the location of a door in the room assigned the priority.The instructions to cause a machine to merge the wall of the first roomand the wall of the second room based on the assigned priority caninclude instructions to cause a machine to join first and second roomsusing the wall from the room assigned the priority. The computer programproduct can also include instruction to receive a user input of apreference of one of the first and second rooms. The computer programproduct can also include instruction to automatically select one of thefirst and second rooms based on one or more features of the wall of thefirst room and the wall of the second room.

This application also relates to a system for searching and matchingrelated vector based objects, drawings and models in a home improvementand design system.

In some aspects, a computer-implemented method for use in generating ahome design layout includes receiving a diagram of a floor plan for aroom, searching a database for other rooms having similar floor plans tothe received diagram of the floor plan for the room, and presenting, viaa graphical interface, the floor plans associated with the rooms havingsimilar floor plans.

Implementations can include one or more of the following.

The method can also include receiving an input of a room type from auser. The room type can be selected from a bedroom, a bathroom, a livingroom, a bathroom, a living room, a kitchen, a dining room, and anoffice. Receiving the diagram of the floor plan for the room can includereceiving a user input sketch of a shape of the room. The user inputsketch can include one or more of the approximate dimensions of the roomand the approximate square footage of the room. Receiving the diagram ofthe floor plan for the room can include receiving a computer-aideddesign (CAD) model of the floor plan for the room. Searching thedatabase for other rooms having similar floor plans can includesearching the database for other rooms having similar placement ofwalls, windows, and door openings. The diagram can be a vector basedinput and searching the database comprises searching for similar vectorbased objects. Presenting the floor plans associated with the roomshaving similar floor plans as the room can include presenting the floorplans according to an order of relevance. The method can also includereceiving a date range input from the user Searching the database caninclude searching the database for similar floor plans generated withinthe date range.

In some aspects, a computer program product is tangibly embodied in aninformation carrier for executing instructions on a processor. Thecomputer program product is operable to cause a machine to receive arequest to receive a diagram of a floor plan for a room, search adatabase for other rooms having similar floor plans to the receiveddiagram of the floor plan for the room, and present, via a graphicalinterface, the floor plans associated with the rooms having similarfloor plans.

Implementations can include one or more of the following.

The computer program product can also include instruction to receive aninput of a room type from a user. The instructions to receive thediagram of the floor plan for the room can include instructions to causea machine to receive a user input sketch of a shape of the room. Theinstructions to receive the diagram of the floor plan for the room caninclude instructions to cause a machine to receive a computer-aideddesign (CAD) model of the floor plan for the room.

In some aspects a, system for use in generating a home design layout isconfigured to receive a diagram of a floor plan for a room, search adatabase for other rooms having similar floor plans to the receiveddiagram of the floor plan for the room, and present, via a graphicalinterface, the floor plans associated with the rooms having similarfloor plans.

Implementations can include one or more of the following.

The system can be further configured to receive an input of a room typefrom a user. The system can be further configured to receive a userinput sketch of a shape of the room. The system can be furtherconfigured to receive a computer-aided design (CAD) model of the floorplan for the room.

In some aspects, this application relates to generating a home designlayout. In other aspects, this application relates to scraping andextracting information from websites and converting this informationinto 2D CAD (computer-aided design) drawings and 3D CAD models. In otheraspects, this application relates to scraping and extracting images andinformation from websites and converting this information into texturemapping objects for use with 3D models, computer generated graphics andphotographs. In other aspects, this application relates to separating afloor plan in a database into separate rooms. In other aspects, thisapplication relates to joining adjacent walls of two distinct rooms. Inother aspects, this application also relates to searching and matchingrelated vector based objects, drawings and models in a home improvementand design system.

The foregoing methods may be implemented as a computer program productcomprised of instructions that are stored on one or more informationcarriers, and that are executable on one or more processing devices. Theforegoing methods may be implemented as an apparatus or system thatincludes one or more processing devices and memory to store executableinstructions to implement the method. A graphical user interface may begenerated that is configured to provide a user with access to and atleast some control over stored executable instructions to implement themethod.

The details of one or more examples are set forth in the accompanyingdrawings and the description below. Further features, aspects, andadvantages are apparent in the description, the drawings, and theclaims.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of a web interface for home design.

FIG. 2 is a diagram of a web interface that includes home design productbookmarks.

FIG. 3 is a diagram of an exemplary home design product bookmark.

FIG. 4 is a flow chart of a process for generating home design productbookmarks.

FIG. 5 is diagram of a web interface for viewing bookmarks of homedesign products.

FIG. 6 is a diagram of a web interface for a home design product.

FIG. 7 is a diagram of a web interface that includes a menu forselecting product information.

FIG. 8 is a diagram of a web interface that includes a home designproduct bookmark.

FIG. 9 is a flow chart of a process for updating information about ahome design product.

FIG. 10 is a diagram of a two dimensional room layout.

FIG. 11 is a flow chart of a computer-aided design (CAD) model selectionprocess.

FIG. 12 is a diagram of a web interface with a CAD model selectionwindow.

FIG. 13 is a flow chart of a model update process.

FIG. 14 is a diagram of a web interface with a two-dimensional roomlayout.

FIG. 15 is a diagram of a web interface with a two-dimensional roomlayout.

FIG. 16 is a diagram of a web interface.

FIG. 17 is a diagram of a web interface with a three-dimensional roomlayout.

FIG. 18 is a flow chart of a texture mapping process.

FIG. 19 is a diagram of a web interface and a menu for selecting productinformation related to a home design product.

FIG. 20 is a diagram of a web interface that includes athree-dimensional room layout.

FIG. 21 is a diagram of a web interface that includes a menu forselecting product information related to a home design product.

FIG. 22 is a flow chart of a texture mapping update process.

FIG. 23 is a web interface that includes a two-dimensional room layout.

FIG. 24 is a web interface that includes a two-dimensional room layout.

FIG. 25 is a web interface with a two-dimensional room layout.

FIG. 26 is a flow chart of process for searching for related roomdesigns.

FIG. 27A is a CAD model of a room.

FIG. 27B is a sketch of a room.

FIG. 28 is a web interface.

FIG. 29 is a selection menu.

FIG. 30 is a flow chart of a process for separating adjoined rooms in afloor plan into separate, non-adjoined rooms.

FIG. 31 is a diagram of a floor plan.

FIG. 32 is a diagram of a room from the floor plan of FIG. 31.

FIG. 33 is a diagram of two rooms.

FIG. 34 is a diagram of the two rooms of FIG. 33 after being joined.

FIG. 35 is a flow chart of a process for joining two rooms,

FIGS. 36A-36E show diagrams of floor plans.

FIGS. 37-59 are diagrams of web interfaces for home design.

FIG. 60 is a diagram showing two-dimensional room layouts.

FIGS. 61-65 are diagrams of web interfaces for home design.

FIG. 66 is a flow chart of a home design process.

FIGS. 67-73 are diagrams of web interfaces for home design.

FIGS. 74-77 are diagrams of graphical images of home design products.

FIG. 78 is a diagram of a CAD model.

FIGS. 79 and 80 are diagrams showing example main object planes.

FIG. 81 is a diagram of a CAD model.

FIGS. 82-84 are diagrams of a graphical image of a home design product.

FIG. 85 is a diagram of a feature level plane.

FIG. 86 is a diagram of a CAD model.

FIG. 87 is a diagram of a feature level plane.

FIGS. 88-90 are diagrams of CAD models.

FIGS. 91A and 91B are diagrams of a feature level plane.

FIGS. 92-95 are diagrams of CAD models.

FIG. 96 is a CAD model of a range.

FIG. 97 is a diagram of a web interface.

FIG. 98 is a diagram of a web interface with a CAD model editing window.

FIG. 99 is a flow chart of a process for use in building athree-dimensional representation of a home design product.

FIG. 100 is a block diagram of a computer system.

Like reference numerals in different figures indicate like elements.

DETAILED DESCRIPTION

FIG. 1 shows a webpage 10 for a home design tool. The home and/orlandscape design tool is a computer software based tool that enables auser to collect information about home and/or landscape design productsfrom websites on the internet and use the collected information fordesigning a room or multiple rooms. The home and landscape design tool(referred to herein as the home design tool) includes a collect mode(indicated by collect button 12), a create mode (indicated by button14), and a collaborate mode (indicated by button 16). In the collectmode, a user collects information and images about home and landscapedesign products from websites. The home design tool stores theinformation in the form of visual bookmarks that are saved in a databaseand can be viewed by the user to compare various products. In the createmode, a user generates a design for a particular room or home by drawingit from scratch, using similar user designs as a starting point orsnapping together existing rooms from the system. Within create mode,the user may add home and landscape design products to their designsusing the home and landscape design products from the visual bookmarksgenerated in the collect mode and associating them with 2D/3D models inthe system. In the collaborate mode, a user can share designs he/she hasgenerated with others and/or can view designs that have been generatedby others.

FIG. 2 shows a webpage 40 associated with the collect mode of the homedesign tool that is accessed by selecting the collect button 12. Webpage40 is used in the collect mode of the home design tool to view, manageand edit visual bookmarks 60 based on home design product selectionsreceived from a user. Webpage 40 includes two visual bookmarks 60 and61. The visual bookmarks provide a user with information about aparticular product in an easily readable format. Visual bookmarks caninclude various information about the home design product that may beuseful in the user when determining whether or not to use the homedesign product in a particular room design. Exemplary information thatmay be included in the visual bookmark or stored in the database relatedto the visual bookmark (e.g., and not included in the informationdisplayed to the user in the visual bookmark) includes an image of theproduct, manufacturer name, vendor name, the name of the products, themodel number, the model type, the size of the product, the finish andcolor of the product, the pattern, the price, the style, theconstruction type, the material(s), the features, mounting options,configuration options, installation type, accessories, accessory styles,application options, detail options, shape, power requirements, andcontrol options.

FIG. 3 shows an exemplary visual bookmark 60 for a kitchen sink.Bookmark 60 includes an image of the sink 70, the name or model of thesink 72, a short summary of the features of the sink 74, the size of thesink 76, and the list price for the sink 78. The information and imageincluded in the visual bookmark 60 is obtained by the home design toolfrom the product website.

FIG. 4 shows a process 90 for generating visual bookmarks based on homedesign products selected by a user from a website. To get started, theuser registers to use the system and either installs the toolbar withintheir Web browser or right clicks on a link within a webpage that allowsthem to add a link to the website to their Favorites list within theirWeb browser. The user generates the visual bookmarks by finding productson the Internet that are of interest to the user (92). The user may usea search engine to locate products of a particular type and/or the usermay browse a manufacturer's website in order to select home designproducts. Once the user has located a home design product of interest,the user selects a collect button from their Internet browser (or fromthe web page if the manufacturer has incorporated the Add to DesignInbutton within their website) to initialize a collection process thatgathers information about the selected product used to generate a visualbookmark (94).

After the user has selected the product for which to generate the visualbookmark, the home design tool extracts information about theuser-selected home design product from the website on which the productis displayed (96). In some implementations, the home design toolextracts the information by comparing what information was previouslychosen to be extracted by other users that bookmarked the same page. Thehome design tool performs an initial product categorization based on theextracted information (98). The home design tool uses the initialcategorization to extract information product specific information fromthe website (104). In some implementations, the type of informationextracted is based on the product categorization. For example, theinformation extracted for a kitchen sink may differ from the informationextracted for cabinets or floor tiling. The product categorization andthe product information extracted by the home design tool are presentedto the user for review (106). The user reviews the categorization andother extracted information and either confirms the information isadequate or provides updated information about the home design product.The home design tool receives the confirmation or updated informationabout the home design product from the user (108) and generates a visualbook mark for the selected home design product based on the extractedcategory and information (110).

FIGS. 5-9 show exemplary screenshots from the process for generating avisual bookmark. As shown in FIG. 5, when the user first uses the homedesign tool, the user will not have any visual bookmarks. In order togenerate the visual bookmarks the user navigates to websites that haveproducts of interest. For example, as shown in FIG. 6, the user maydesire to locate a kitchen sink. In this example, the user has selecteda Kohler® sink 90 from the manufacturer's website 124. In order togenerate a visual bookmark, the user selects the collect button 128 ontoolbar 126. The home design tool extracts information related to thecategorization of the sink 90 as well as other information that may beincluded in the visual bookmark. Since the information is automaticallygathered by the home design tool, errors could potentially exist in thegathered information. In order to correct any errors, the informationextracted about the sink is presented to the user for verification. Forexample, in some implementations, e.g., as shown in FIG. 7, theextracted information is presented in a pop-up window 140 for the userto review. In this example, the home design tool has extractedinformation about category, product type size, color, and price as shownin boxes 144, 146, 148, 150 and 152 respectively. If any of thisinformation is incorrect, the user can update the information byinputting the correct information into the respective box. After theuser has confirmed the information, the home design tool uses theextracted information to generate a visual bookmark. For example, asshown in FIG. 8, a visual bookmark 162 for the sink includes an image ofthe sink as well as the information about the size, price, and generaldescription of the sink extracted from the website.

This visual bookmark generation process is based on the integration ofweb scraping and extraction technology. The home design tool thatexecutes the bookmark generation process is a hybrid taxonomy andfolksonomy based system built for bookmarking anything related to thehome improvement industry including products, design ideas and articles.In general, folksonomy is an Internet-based information retrievalmethodology comprised of collaboratively generated, open-ended labelsthat categorize content on the web. In contrast, a taxonomy based systemis a hierarchical tree structure of classifications for a given set ofobjects. Taxonomy based system use a pre-established, controlledvocabulary.

The home design tool combines aspects of both taxonomy and folksonomy.For example, the categorization of a home design product selected by auser is categorized into pre-defied categories of product types and,thus, relies on a taxonomy based system. It is believed that using ataxonomy based system to categorize the home design product can providevarious advantages such as offering a standard categorization schemathat allows users to browse a directory of public visual bookmarks,simplifying product searches, simplifying and/or automating the processof associating the correct 2D drawing and 3D models with the bookmark,identifying key characteristics of the product and/or using this torefine the properties of the associated 2D drawing and 3D model. Whilethe categorization of the product is based on a taxonomy based system,the information included in the visual bookmark for the home designproduct is based on a folksonomy based system. As such, the informationin the visual bookmark is gathered from the information available on thewebsite of the selected home design product. The tags for the visualbookmark are generated using a folksonomy based system to furtherdescribe the product characteristics including style, color, anddimensional information. It is believed that using a folksonomy basedsystem to for collecting and displaying additional information about thehome design product can provide various advantages such as offering amore open ended system for categorizing products based on more detailedinformation such as the finish and color of the product, the pattern,the style, the construction type, the material(s) and other optionslisted above, offering a tag based system to simply and refine searchesfor visual bookmarks created by other users; offering complementarysystem to the taxonomy based directory of public visual bookmarksallowing users to refine their product browsing with tags; and/orproviding a more detailed information for associating the correct 2D CADdrawing and 3D CAD model with a visual bookmark.

The taxonomy based categorization and folksonomy based tags can be usedto build intelligence for the home design tool. As users collectproducts, design ideas, and articles, the home design system uses aneural network to learn and identify the correct taxonomy basedcategories and folksonomy based tags.

FIG. 9 shows a process 170 for updating and providing more accurateproduct categories and tags for a product based on previous informationcollected by the home design tool for the product. As described above,the home design tool automatically pre-selects both the productcategorization and the product information and presents thecategorization and information to the user who can confirm or update thecategorization and information. When the user confirms or updates thetaxonomy-based categorization for a particular home design product, thehome design system records the user's input (172). Similarly, when theuser confirms or updates the folksonomy-based information for theparticular home design product, the home design system records theuser's input (174). The system uses the information recorded about theproduct categorization and product information to provide more accurateautomated categorization and product information for subsequent users(176).

For example, in FIG. 7, the home design system automatically categorizedthe sink as a home product (as shown in box 144). In addition, the homedesign tool extracted various information about the sink such as theproduct type (as shown in box 146), the size (as shown in box 148), thecolor (as shown in box 150), and the price (as shown in box 152). If anyof this information is incorrect, the user can modify the information.For example, if the price of the sink were $250 rather than $542.25 asextracted by the home design tool, the user could enter the correctprice of $250 into box 152. The home design tool would record the changein the price for the home design product made by the user. The visualbook marks are stored in a database and available for others to view.Subsequently, if another user selects the same sink, the home designtool could use this information to initially provide the correct priceof $250 to the user.

In some examples, the user may modify information such that the correctinformation is changed to be incorrect. For example, in FIG. 7 the sizeof the sink is accurately shown as 33×22. However, the user may changethe size of the sink to be 12×12 or any other incorrect dimension. Thismay cause the home design tool to subsequently display the informationentered by the user (e.g., 12×12) rather than the correct informationextracted from the website. However, when this happens, the user willcorrect the information. As such, over time, the responses of the usersof the system will show that the correct size of the sink is 33×22 andthe system will “learn” to display this information for the sinkdimensions in spite of the incorrect information entered by one of theusers.

In some implementations, the home design system can use neural networksas described to follow. The system scans the page for the initialtaxonomy based criteria. Once identified the system uses the taxonomybased criteria to scan a database of words and phrases identified to bekey characteristics of the taxonomy based product type. The system alsouses the taxonomy based criteria to scan the database of tags used byprevious users to identify not only a particular page/product but tagsused to describe bookmarks within the same taxonomy based productcategory. The system uses these words to scan the page for these phrasesand words and presents the user with a list of potential tags forfurther folksonomy based categorization of the product.

Referring back to FIG. 2, the home design tool groups the visualbookmarks according to product category and product type. For example,webpage 40 includes a menu 41 of collections of visual bookmarksgenerated by the user. The visual bookmarks are organized intocategories based on the product categorization and into subcategoriesbased on the type of the product. Exemplary product categories includeappliances (e.g., as indicated by arrow 44), kitchen cabinets (e.g., asindicated by arrow 46), plumbing fixtures (e.g., as indicated by arrow48), flooring, appliances; building structure; cabinets & storage;floors, walls & counters; furniture & accessories; landscape & outdoor;lighting; plumbing and other. Each of these product categories can besub-divided into particular types of products. An exemplary subdivisionof product categories into particular types of products is shown tofollow:

Appliances

-   -   Cooktops    -   Dishwashers    -   Laundry    -   Microwave Ovens    -   Ovens    -   Ranges    -   Refrigerators    -   Other

Building Structure

-   -   Columns    -   Doors    -   Fireplaces    -   Railings    -   Stairs    -   Windows    -   Wood. Stoves    -   Other

Cabinets & Storage

-   -   a Closets    -   Kitchen Cabinet    -   Bathroom Cabinet    -   Pantry    -   Wine Storage

Floors, Walls & Countertops

-   -   Architectural Elements    -   Carpeting    -   Ceilings    -   Countertops    -   Floor Laminate    -   Hardwood Flooring    -   Paint    -   Rugs    -   Tile    -   Vinyl    -   Wallpaper    -   Window Coverings    -   Other

Furniture & Accessories

-   -   Bedding & Linens    -   Beds    -   Chairs    -   Decorative Accessories    -   Decorative Hardware    -   Desks    -   Fabrics    -   Kitchen Accessories    -   Ottomans    -   Plants—Indoor    -   Sofas    -   Tables    -   Tableware    -   Other

Lighting

-   -   Lamps    -   Ceiling—Attached    -   Wall—Attached

Landscape & Outdoors

-   -   Arbors    -   Columns    -   Fences    -   Hot Tubs    -   Outdoor Furniture    -   Patio Stones    -   Plants    -   Pools    -   Spas    -   Trees    -   Wall Stones    -   Other

Plumbing

-   -   Bath/Showers    -   Bathtubs    -   Faucets    -   Hot Tubs & Spas    -   Sinks—Bathroom    -   Sinks—Kitchen    -   Saunas & Steam Baths    -   Showers    -   Toilets    -   Other

Other

-   -   Heating & Cooling    -   Art

For example, the category of plumbing fixtures can include productstypes such as faucets, sinks, and tubs (e.g., as indicated by arrows 50,52, and 54 respectively). It is believed that arranging the visualbookmarks into product categories and product types can provide variousadvantages such as making the information more easily accessible for theuser to search for these products or browse a product directory based onthese types of categories.

Webpage 40 also includes information about products not selected asvisual bookmarks by the user. For example, webpage 40 includes similaruser collections 62. The similar user collections 62 include other itemsthat are similar to the items selected by the user. The user can accessthe similar user collections 62 to view other products without having tosort through the large volumes of information available on the internet.The home design tool determines home design products to include in thesimilar user collections based on the products for which the user hasgenerated visual bookmarks. For example, as shown in FIG. 2, the userhad generated bookmarks for various dual-basin sinks. As such, thesimilar user collections could include dual-basin sinks available fromother manufacturers and selected as visual bookmarks by other users. Thesystem uses the taxonomy based product categorization and the folksonomybased tags to identify similar products.

Webpage 40 also includes sponsored links 64. In general, sponsored links64 are links to products by a particular manufacturer associated withthe type of products that the user has bookmarked. The manufacturer maypay a fee to have the sponsored links 64 displayed in order to advertisetheir brand or their particular products.

Webpage 40 also includes text based web advertisements 66. Theseadvertisements can be sponsored by various companies or organizations.The displayed advertisements are selected by the home design tool basedon the information about the type of products the user has generatedvisual bookmarks for. For example, in FIG. 2, the user had generatedvisual bookmarks for various types of sinks and the informationpresented in the web advertisements 66 is related to the generalcategory of sinks. Both the text and image based advertisements mayinclude similar products using the same taxonomy based productcategorization and the folksonomy based tags to identify similarproducts in the advertising inventory. In some implementations,advertisers can choose product categories and products tags required fortheir advertisements to appear in the system.

The taxonomy based categorization, the folksonomy based tags and useractivity tracking offer the foundation for the system's product searchalgorithms. Users may type a product name and characteristics into thesearch field. The system identifies and prioritizes the most popularproduct bookmarks for the specific terms based on what productcategories and tags they assigned to the product, how many users havebookmarks these products, how many users have incorporated theseproducts within their designs and how many users have viewed theseproducts in other users collections and within other users designs. Thealgorithm relies heavily on how many users incorporate the bookmark intotheir designs since this shows that the user may actually purchase theproduct. Advertisements may also appear in search results using similarcriteria as bookmarks. If the advertisers assign a specific URL to anad, the system will include both visual bookmarks and advertisementselections for the same product as part of the prioritizationcalculation for the products. After typing a search term into the searchfield, specifying that the users is searching for products, and clickingthe search button, the system presents the user with results based onthe algorithm outlined above. On the left side of the screen, the useris presented with options to refine the search. For example if a userstypes in two basin sink within the search box, the results appear and onthe left side of the system automatically assigns a product category ofplumbing and a product type of sink. The system may also present theuser with the product style of two basin as a taxonomy basedcategorization or as a folksonomy based tag. Tags extracted from andmost commonly shared among the search results are presented to user.These tags are shared characteristics of the product as outlined abovein the description of potential tags. The user may select these tagssuch as “self-rimming” (an installation type) to refine the searchresults to sinks that are self-rimming meaning they are installedthrough the top of a counter. The user may also refine the importance ofsearch criteria such as price, size, and specific tags. Specific tagsmay be selected from a list to be added to a search slide bar. Users mayadjust the search sliders to adjust the importance of each criteria.These importance parameters are used as coefficients in search algorithmto raise or lower the weight of different metrics in search results.

As described above, the home design tool allows a user to collectinformation about home design products and use the information fordesigning a room or multiple rooms. In the create mode, a user cangenerate a design for a particular room or rooms using the home designproducts collected in the collect mode. In order to allow users to addthe products for which they generated visual bookmarks into a roomdesign, the home design tool uses information collected in the collectmode about a home design product to associate and modify two-dimensionaland three-dimensional computer aided design (CAD) models from thesystem's database to the specifications of the product.

FIG. 10 shows an exemplary CAD design 170 that includes multiple CADmodels for various homes design products. For example, CAD design 170includes a CAD model 172 for a stove, a CAD model 174 for a sink, a CADmodel 176 for a set of cabinets, and a CAD model 178 for a table. Thehome design tool generates the CAD models for home design products basedon information extracted from the website.

FIG. 11 shows a process 180 for generating CAD models for home designproducts presented on a webpage based on a user selection of the homedesign product. The home design tool receives a user selection of a homedesign product from a website (182) and extracts information about thehome design product (184). The home design tool presents productcategorization information to a user based on information extracted fromthe website (186) and the user either confirms or updates the productcategorization information (188). The home design tool also presentsproduct information to the user (190) and receives a confirmation or anupdate of the information from the user (192). The extraction andconfirmation of information (e.g., steps 182, 184, 186, 188, 190 and192) are similar to the process described above in relation togeneration of the visual bookmark for a home design product. The homedesign tool identifies potential two-dimensional and/orthree-dimensional model(s) that are potential matches for the homedesign product based on the extracted categorization and informationabout the home design product (194). More particularly, the home designtool includes a database that includes multiple, different standard CADdrawings for different home design products. The home design tool usesthe categorization information for the product and the dimensionalinformation to narrow the number of potentially matching CAD models forthe home design product. The potentially matching CAD model(s) arepresented to the user (196). The user can select the CAD model that mostclosely represents the product he/she selected from the website. Forexample, as shown in FIG. 12, the home design tool presents the userwith a pop-up window 220 that includes multiple CAD model choices 226,230, 234, and 238. The user can select a particular CAD model from thedisplayed CAD models 226, 230, 234, and 238 by selecting the associatedbutton 224, 228, 232, or 236, respectively. Once the home design toolreceives the user selection of a particular one of the CAD models (198),the home design tool associates the CAD model with the home designproduct and the visual bookmark for the home design product (200). Nextthe system will present the user with specific properties from theoriginal visual bookmark for the 2D/3D model (202). For example, thecolor white and the sizing options of 33″×22.″ The user will confirmthese options and the system updates the model (204). For example inthis case the system applies the color white to the two basin sink andparametrically adjusts the base model to become 33″×22″, Non-parametricadjustments may also be made. Instead of parametrically adjusting amodel, the system may simply replace the model with a model that meetsthe specific criteria.

An additional implementation may include interpreting an image andtransforming the image into a 3D model. This will be used for productsthat may not be included in the site's database of CAD models. Forexample if the user bookmarks a lamp and the system does not have a lampin the database, the system will prompt the user to create a 3D model ofthe image. For a tall floor lamp, the system would identify the edges ofthe object and prompt the user to verify that the edges are correct orincorrect. The user can select nodes to move the vector based shapearound the image and identify the shape of the image. The user may alsodraw a closed shape around objects. Once identified, the user may selecthow they would like the system to convert the image into 3D. The usercould browse the directory of CAD model in the system, create her owndesign or use the image as a basis for the CAD model. For example with atall floor lamp, the system would identify the edges of the lamp andremove other graphics from the picture. The user could then use therevolve tool to revolve the image of the lamp around a vertical axis tocreate a 3D version of the product. The cropped image also becomes atexture map that may be applied to the 3D model. Tools used for creatinga model from scratch may include: lines, arcs circles, rectangles,polygons, 3D box, spheres, cylinder, cones, toruses, wedges, revolvetools, sweep tools, 3D poly-lines, 3D splines, and other standard CADdrawing tools.

It is believed that generating CAD models for images found on theinternet can provide various advantages. For example, since the homedesign tool makes the entire Web a content source for two-dimensionalCAD drawing and three-dimensional CAD models of products, the users ofthe home design tool can select and use any home design product indesigning a layout and is not constrained to manufacturers that offerCAD models for their products in specific formats. The user is notconstrained to manufacturers that provide CAD models because the CADmodel matching and modification process identifies key characteristicsof products based on information available on the home design product'swebsite and converts this information into CAD models that represent thestyle, color, and dimensional information of the product.

The taxonomy based categorization and folksonomy based tags used toassociate the CAD models with the home design product selected by theuser build the intelligence for the system, which helps the system tocorrectly identify which CAD drawings and models to associate with aparticular home design product. As described above, when a user addsthis visual bookmark to a CAD drawing, the system presents the user witha selection of the closest matching CAD files (two-dimensional and/orthree-dimensional CAD models) to associate with the visual bookmark.FIG. 13 shows a process 212 for building intelligence and using theintelligence about selected CAD design choices to provide more accurateselections to a user. As users associate the various CAD models/drawingswith a specific Web page and product information, the system recordswhich CAD model is selected the most frequently (214). The system alsorecords what properties from the visual bookmark like size and color areused to modify the model in the database (215). The home design tooluses this recorded information to update a ranking of the CAD models andprioritizes CAD model choices presented to the user for future bookmarksaccordingly, as well as update the properties of the presented CAD model(216).

FIGS. 14-16 show an exemplary use of CAD models in a home design. OnceCAD models have been associated with a particular user-selected product,the home design tool displays the home design products for the user touse in designing a layout for a room. For example, FIG. 14 shows ascreenshot 240 of a web interface for home design. The web interface 240includes a portion 241 that shows pictures of the home design productsfor which the user has generated visual bookmarks and CAD models. It isbelieved that showing the image of the product (e.g., images 242 and248) as opposed to showing the CAD model associated with the product canaid the user in easily recognizing and selecting the product. The webinterface 240 also includes a portion 249 that provides a way for theuser to access supplier generated CAD files. The supplier CAD files arefiles provided by the manufacturer. The user can sort for a supplier CADfile by entering information about the type of home design producthe/she is looking for.

The home design tool allows the user to insert home design products intoa home design layout (e.g., layout 246) by dragging the image of theproduct from section 241 into the desired location. For example, in FIG.14, the user has selected the image of sink 242 and placed the sink intothe layout 246 (as indicated by sink 244 in the layout 246). After theuser has placed the home design product into the layout 246, the homedesign tool replaces the image of the home design product with theassociated CAD drawing of the home design product (e.g., as shown by CADmodel 250 in FIG. 15).

In some implementations, once placed the object remains selected, and aproperties palette on the right side of the screen appears, allowing theuser to confirm the size and color of the sink or change it and see theproperties update. In some additional implementations, the user selectsthe visual bookmark to be inserted and the properties palette appearsprompting the user to verify the correct options are selected such ascolor and size. Once the user selects the properties, a 2D/3D model isparametrically generated by automatically modifying the original modelin the database to the properties specifications from the originalbookmark or based on user input. The user may now drag the 2D/3D modelinto his/her design.

As shown in FIG. 16, after the home design tool has replaced the imageof the home design product (e.g., as shown in FIG. 14) with the CADmodel of the home design product (e.g., as shown in FIG. 15), the usercan view information relating to the home design product by moving acursor over the CAD model 250. For example, as shown in FIG. 16, whenthe user positions their mouse over the CAD model 250 a message 252appears that includes the visual bookmark information associated withthe CAD model and the selected home design product. It is believed thatthis can be beneficial because the user can easily view information andan image of each home design product that has been added to a room'slayout.

While in the implementations described above in relation to FIG. 11, thehome design tool associates the CAD model with a product selected by theuser at the time the visual bookmark is generated, other implementationsare possible. For example, in some implementations, the potential CADmodels may be presented to the user when the user attempts to add thebookmarked home design product to a layout. Associating the CAD modelwith the home design product at the time the product is added to a roomlayout can provide the advantage of requiring the user to select a CADmodel only for the products they choose to add to the layout rather thanfor each product for which the user generates a visual bookmark.

While in the implementation described above, the system associates a CADmodel with a user-generated bookmark, other implementations arepossible. In some implementations, the user specifies a generic 2D CADdrawing/3D CAD model to place in their design. The system scans thedatabase of visual bookmarks that other users have matched to thespecific CAD model and presents the user with a prioritized list ofvisual bookmarks. The system also scans the database of the entiredatabase of advertisements that have CAD models associated with theadvertisements. The system also presents the user with a prioritizedlist of image and text based products advertisements. The user thenselects which visual bookmark or advertisement to associate with the CADmodel.

FIG. 17 shows a web interface 260 that includes a three-dimensional view262 of a room based on a layout generated using two-dimensional CADmodels. The three-dimensional view 262 allows a user to view the homedesign products and the placement of the products within the room. Thehome design tool includes two dimensional and three dimensionalrepresentations for most objects. In 3D, the system displays the 3Drepresentation of objects included in a CAD drawing to generate objectshaving a visual appearance in 3D using auxiliary information fromdatabase and minimal user input based on objects' properties that arespecified in the two dimensional CAD drawing. The constructedrepresentation includes polygons (triangles), which are displayed in 3Dmode using standard graphics pipeline with parallel projection. The setof triangles is generated for rendering without special hardwareacceleration and the rendering can be implemented using graphicstechnology (e.g., Macromedia Flash). Some implementations may usehardware acceleration to provide more realistic and more detailedgeometry model rendering.

In order to provide the user with a visual representation of the floorand other surface coverings, the home design tool performs texturemapping of a product selected from a website based on an image extractedfrom the website. It is believed that providing texture mapping based onan image selected from the internet can provide various advantages suchas allowing a user to view any product they find displayed on theinternet rather than limiting the user to manufacturers who haveprovided texture mapping materials or models with texture mapsassociated with them.

FIG. 18 shows a process 280 for generating and applying texture mappingof a texture shown in an image based on a user selection of the homedesign product exhibiting the texture. The home design tool receives auser selection of a product (e.g., a flooring, wall covering, or otherproduct that has a texture to be applied to a surface) from a website(282) and extracts information about the product (284). The home designtool presents product categorization information to a user based oninformation extracted from the website (286) and the user eitherconfirms or updates the product categorization information (288). Thehome design tool also presents product information to the user (290) andreceives a confirmation or an update of the information from the user(292). The extraction and confirmation of information (e.g., steps 282,284, 286, 288, 290 and 292) are similar to the process described abovein relation to generation of the visual bookmark for a home designproduct. In some implementations, the same extraction process can beused to extract the information for the texture mapping as for thevisual book-marking and CAD model association. In order to obtainadditional information about what is shown in the image that is notavailable as text associated with the image on the webpage, the homedesign tool prompts the user to enter image specific information (294).Examples of image specific information include the number of tiles shownin an image, the number of slats of hardwood shown in an image, theexpected size of a non-regularly repeating texture such as carpeting ormarble. The image specific information can also include informationabout how the texture might wrap around a sample object, what theparameters are for wrapping and projecting around an object, imageclipping options, reflection properties, illumination mapping, scale,warping options and/or bump mapping options allowing the texture to notonly be applied to surface(s) but allowing the texture to have detailedcontours.

In some implementations, the system prompts the user with a preview ofwhat the texture map might look like and automatically crops the imageto make a consistent texture map or prompt the user to crop the image tomake a consistent texture map. The system can also include hatchingpatterns to select firm that represent how the texture maps. For examplefor 12″ tile, the user would select a hatch pattern named 12″ that haslines spaced about 12⅛″ apart vertically and horizontally. The ⅛″ allowsfor grout. The user may overlay the hatch pattern over the image andfurther align the hatch to the image. This is turn defines how the imagewill be repeated when mapped onto a surface. Another option is tointerpret the image directly from the website and use a combination ofthe taxonomy based categorization, folksonomy based tags and neuralnetwork based learning to identify what type of repetitivecharacteristics and other properties of a texture. Then the informationwould be used to explore whether the texture meets these properties bycomparing the pixels of the image with a vector based hatch pattern suchas a grid defining a tile or a pattern, in some implementations, thesystem can use the pixels of the image to identify the correctrepetitive characteristics of the image. In order to do so theapplication will perform vectorization of the image using edge detectionalgorithms (i.e. Canny algorithm based on intensity jumps detection). Ifthe vectors detected during this processing can be treated as texturemapping direction lines, then they are suggested to the user as anoption for texture mapping.

The system may also include sample objects for defining how the texturewill map around surfaces like couches and chairs. For example, astripped fabric could map in multiple directions to a couch. The systemwill provide tools for toggling how the texture maps to sample objectswhich in turn will define how the texture maps to 2D/3D CAD models inthe design. The home design tool uses the information extracted from theproduct website in addition to the information received in response tothe user prompt to map the texture to a surface by replicating the imageto cover the surface and wrapping the image around a surface if required(296).

FIGS. 19 and 20 provide an exemplary texture mapping for a tile flooringbased on an image of some tiles collected from a website. As describedabove, when the user selects a particular image of the tiles, the homedesign tool generates a pop-up menu 313 that prompts the user to providefeedback about the product and about the selected image of the product.Some information about the product such as the expected category (e.g.,as indicated in block 314), product type (e.g., as indicated in block316), and size (e.g., as indicated in block 318) is provided by the homedesign tool searching the webpage for the relevant information. The userviews and confirms this information. Other information, such as imagespecific information, is not likely to be included in the text of thewebpage. Therefore, the home design tool prompts the user to provide theinformation. As shown in FIG. 19, the home design tool prompts the userto enter the number of tiles shown in the image (e.g., as indicated inblock 320). In this example, the user would input the number two forrows and two for columns since there are four tiles shown in theselected image. The automatic edge detection algorithm processes theimage and detects the edges in the image and provides initial numbers (2and 2) for the user to accept the input.

When the user goes into create mode, after receiving the informationfrom the user, the home design tool maps the texture to a surface. Forexample, as shown in FIG. 17, the user could map the tile texture ontothe floor 264 of the three-dimensional view of the room 262. Forexample, if the room 262 was ten foot by ten foot, the home design toolcould determine that the size of the image corresponded to a two foot bytwo foot (e.g., each tile is 12 inches by 12 inches and there are fourtiles). As such, as shown in FIG. 20, the home design tool could sizethe image appropriately to replicate the image 25 times (e.g., 5 rowsand 5 columns) to produce the effect of the texture appropriately sizedfor the surface.

While in the example above, the amount of area to be covered by onerepetition of the image was discernable based on the size of the tileand the number of tiles shown in the image, in some examples the size isnot easily determined from the image. For example, as shown in FIG. 21,a user may select a non-regularly repeating texture such as carpeting.In this example, the user has selected an image 340 of a carpet from acarpet manufacturer's website 342. The home design tool generates apop-up window 344 requesting input from the user. The home design toolautomatically populates the category and product type for confirmationby the user (e.g., as shown in boxes 348 and 350). Since it may bedifficult for the user to identify the expected scale of the carpetshown in the image, the home design tool displays various representativesized images of a similar type of carpet. For example, as shown inwindow 344, the home design tool presents a near view image of thecarpet 354, a moderately sized image of the carpet 358, and a large areaimage of the carpet 362. The user selects the appropriate scale of thecarpet in the image 346 by comparing the image 346 to the differentlysized images 354, 358, and 362 and selecting the appropriate button 352,356, 356, or 360.

The taxonomy based categorization and folksonomy based tags build theintelligence for the system, helping it identify which tag prompts todisplay for specific URLs and assist in mapping textures properly. Asshown in FIG. 22, as users associate the various properties with images(textures) collected from a specific Web page, the system records theinformation (342) and learns which properties are selected the mostoften and prioritizes user choices for texture mapping accordingly(344).

It is believed that generating texture models for images found on theinternet can provide various advantages. For example, since the homedesign tool makes the entire Web a content source for the texture maps,the users of the tool can select and use any carpeting, paint, tiles,hardwood, marble, and the like in designing a layout and is notconstrained to manufacturers that offer texture models for theirproducts. In addition the system provides a search feature for texturescontributed by other users. As outlined above for product searches, thetaxonomy based categorization, the folksonomy based tags and useractivity tracking offers a similar foundation for the system's texturesearch algorithms.

As described above, in addition to enabling a user to collect productinformation (e.g., by creating visual bookmarks) and enabling the userto create room layouts that include the selected product information,the home design tool also allows the user to collaborate with otherusers regarding various home design issues. For example, the user cancollaborate with other users by searching of products that other usershave selected or viewing room designs that other users have generated.In general, when a user creates a product collection or generates afloor plan, the user can select whether or not to make a productcollection or the floor plan available for others to view.

FIG. 23 shows an exemplary floor plan 358 that is shared for other usersof the home design tool to view. It is believed that allowing otherusers to view the design can provide various advantages both to theperson who generated the design and to those who are viewing the design.For example, in some implementations, when others view the design theycan rate the design (e.g., on a scale of 1 to 5 as shown in ratings 352)to provide feedback to the person who generated the room. Those viewingthe design can also provide comments 353 about the design. For example,if another user had previous positive or negative experiences with asimilar room layout or with a particular product they could providethose comments about the design such that the person who generated thedesign would be aware of their positive or negative experience. Theperson who generated the room design can then decide whether or not tomodify the design based on the feedback. It is believed that this typeof information sharing can be beneficial especially to non-professionalusers who are attempting to design a room and may otherwise not haveaccess to such historical information.

When a user shares their design, other users can view not only the CADdrawing of the room but also the home design products that the user hasselected to place within the design. In addition to the CAD design, thehome design tool displays images of the home design products included inthe design. For example, in FIG. 23 the home design tool displays animage 368 of the refrigerator 369 in the CAD drawing 358, an image 370of the tile 361 included in the CAD drawing 358, an image 372 of thesink 373 included in the CAD drawing 358, and an image 374 of thecabinets 375 included in the CAD drawing 358. As shown in FIG. 24, inaddition to providing the images 368, 370, 372, and 374 of the homedesign products the home design tool can allow the person viewing thedesign to see additional information relating to the home design productby moving a cursor over the product in the CAD model 358. For example,when the user positions their mouse over the CAD model 373 of the sink amessage 376 appears that includes the visual bookmark informationassociated with the CAD model for the sink. It is believed that this canbe beneficial because the user can easily view information about a homedesign product that another user has included in the room design withoutsearching for the information themselves.

In addition to viewing room designs generated by other users, in someimplementations, the home design tool allows users to search a databaseof room designs generated by other users to find rooms with similarfloor plans. For example, as shown in FIG. 23, in collaborate mode, thehome design tool provides the user with a list of similar designs 361and the ability to search. The similar designs are designs with similarfloor plans (e.g., similar room shape, room dimensions, windowplacement, door openings and/or door placements) that have beengenerated by other users of the system. Similar designs may also befound by using taxonomy based and folksonomy based tags that users haveassigned to their entire floorplan, individual rooms and products in thedesign. For example, a user may choose, artisan, country, contemporary,hip, or traditional for a design style from the taxonomy based lists ofdesign styles. The user may also choose add folksonomy based tags likebreakfast nook, colonial, island, mud room entrance, pantry to identifysome characteristics of his/her design. Lastly the system may generatesearchable tags based on the characteristics of the designs. For exampleif the user draws a center island in his/her design, the system may tagthe design with words like “island” and “center island”. Similaralgorithms are used to search for entire floor plans.

Designs 362, 364, and 366 are exemplary designs included in a list ofsimilar designs 361 for the room design shown in CAD model 358. Whilethe designs included in the similar designs may not perfectly match thefloor plan of the room designed by the user, they can still providevaluable information about how other users have organized the layout anddesign of a similar room. If a user desires to view one of the designsincluded in the list 361 of similar designs 361, the user selects thedesign from the list and the design is displayed by the home designtool. For example, as shown in FIG. 25, if the user selects design 362,the home design tool would present the CAD model 370 associated withdesign 362 and the products 371, 372, 373, and 374 included in the CADmodel 370 for the user to view. When the user navigates to a designpreviously included in the list of similar designs, the home design tooladds the previously viewed design to the list of similar designs (e.g.,the previously viewed CAD model 358 is included as a design 372 in thelist of similar designs for the CAD model 370).

FIG. 26 shows a flow chart of a process 380 for generating anddisplaying similar designs based on a vector based search. In order tosearch for similar designs, the home design tool received inputregarding the floor plan of a room for which they would like to viewsimilar designs. The system receives the information about of the floorplan from the user (382) and extracts floor plan characteristics fromthe received information (384). Exemplary floor plan characteristicsinclude room shape, room dimensions, window placement, door openingsand/or door placements. The user can provide the information about thefloor plan in various forms.

For example, in one implementation as shown in FIG. 27A, the user canprovide a CAD model 391 of the floor plan. From the CAD design the homedesign tool can extract the location of doors, wall openings windows aswell as the shape and dimensions of the room. In general, vectorgraphics (also referred to as geometric modeling or object-orientedgraphics) is based on the use of geometrical primitives such as points,lines, carves, and polygons, which are all based upon mathematicalequations to represent images in computer graphics. Vector graphicsdiffers from raster graphics, which is the representation of images as acollection of pixels (dots).

As shown in FIG. 27B, in another implementation the user can provideinformation about the floor plan in the form of a sketch 392 of theroom. For example, the user can sketch the room using their mouse orother computer input tool. In the search sketch pad the user can drawwalls and add windows, doors and wall openings. The search algorithmfirst looks for rooms that have a similar shape such as L-shaped orrectangular, Next the system identifies rooms that have a similardimensions for the room walls (closets may be ignored). Next the systemidentifies similar window, door and wall opening locations. In additionthe system compares the users' collections with products/visualbookmarks within the search results and prioritizes them based onsimilarity. The results of the search are based on a set ofheuristically defined metrics for the design and should not necessarilyresult in only exact matches. Close matches are also reported to theuser.

In some implementations, the user can set importance of search criteriasuch as room dimensions; wall, window and wall openings; and similarityof objects in the design to the users collections to find designs withsimilar tastes and dimensional information. These importance parametersare used as coefficients in search algorithm to raise or lower theweight of different metrics in search results.

The taxonomy based categorization of room names, the folksonomy basedtags for assigning criteria to rooms and user activity tracking offerthe foundation for the system's product search algorithms. Users maytype a room name and characteristics into the search field. The systemidentifies and prioritizes the most popular room designs for thespecific terms based on what room categories and tags a user assigned tothe design, how many users have bookmarks these designs, how many usershave incorporated these room designs within their own designs and howmany users have viewed the designs. The algorithm relies primarily onthe shape, size and similar products variables as outlined above. At thesecond level the system relies heavily on how many users incorporateanother user's design into their designs since this shows that thisdesign is popular. Advertisements (entire room designs created byadvertisers) may also appear in search results using similar criteria asto other users' designs. Advertisements that are already incorporated inother designs will also be included in the calculation even if thedesign was edited. After typing a search term into the search field,specifying that the user is searching for designs, and clicking thesearch button, the system presents the user with results based on thealgorithm outlined above. On the left side of the screen, the user ispresented with options to refine the search. For example if a user typesin L-shaped kitchen within the search box, the results appear and on theleft side of the system automatically assigns a room name of kitchen anda tag of L-shaped. The system may also present the user with the designstyles like artisan, country, contemporary, hip, traditional. Thesestyles are taxonomy based characteristics assigned to design by theusers at the time of creation. Additional options for refining searchresults are outline below.

It is believed that allowing a user to search for designs based on asketch can provide the advantage of allowing the user to easily searchfor related floor plans without having to commit the time and effort ofgenerating an entire room design. In addition it offers a uniqueadvertising opportunity for advertisers to offer complete room designsall with brand name products that they manufacture or sell.

As shown in FIG. 29, in another implementation the user can provideinformation about the floor plan by inputting information about the roomplan using a text based user interface 393. For example, the user canenter information about the room type 394, shape 395, and square footage396. In some implementations, in order to better refine the informationabout the room, the home design tool can present the user with exemplaryroom layouts (e.g., layouts 397 and 398) based on the limitedinformation such that the user can select the room design most similarto what they are attempting to design. The exemplary room layoutspresented to the user can be based on room layouts that are frequentlyviewed or are included in the system that match the criteria input bythe user.

FIG. 28 shows how the user can select a room type and then draw walls,windows, doors and door openings in the search sketch pad to find roomdesigns to use in their overall FloorPlan. The user can set importanceof search criteria such as room dimensions; wall, window and wallopenings; and similarity of objects in the design to the userscollections to find designs with similar tastes and dimensionalinformation. Users can drag these rooms into their designs to becomepart of the overall floor plan as outlined below or simply use the roomas a starting point or to gather ideas for their own designs.

For example, the home design tool can receive one or more inputs fromthe user that rank or select properties of room designs that can be usedto sort the search results. This searching allows the user to view roomdesigns that are most similar in shape and/or design to their room basedon the received inputs.

The design tool allows the user to enter the type of floorplan they aresearching for. For example, the user can search for either a room or anentire floor plan (as indicated by arrow 540). In order to select eitherthe room or floor plan as the type of design for which the user issearching, the user clicks on the appropriate button in the home designtool.

The user can also enter a room type that can be used to further limitthe search results presented to the user (as indicated by arrow 541). Byentering the room type, the user can view only rooms that will be usedfor the same function as their intended use of a room. For example, if auser is searching for kitchen designs, it can be beneficial to presentonly kitchens having similar shape and design to the room rather thanpresent bedrooms or other types of rooms. Exemplary room types includebedroom, bathroom, living room, kitchen, study, play room, dining room,and the like.

The home design tool also allows the user to enter the basic shape ofthe design (e.g., as described above in relation to FIGS. 27A and 27B).In some implementations, the home design tool includes a searchsketchpad 542. The search sketchpad is a tool that allows the user toenter the general shape of the room and any doors (using door button543), windows (using window button 544), and other openings (e.g., usingopenings button 545).

In addition to the type of room (as indicated by arrow 541) and thegeneral shape of the room entered in the sketchpad 542, otherinformation can be used to further limit the search results. Forexample, information about the importance of various search criteria canbe entered, for example, based on a slider bar for each type ofinformation that ranks the importance of that feature. The home designtool uses the information about the importance of the various featuresto filter the search results. As shown in FIG. 28, the user can selectthe amount of relevance the home design tool places on the dimensions ofthe room (as indicated by slider bar 548). If the slider bar is moved toindicate a greater emphasis on the dimensions of the room (e.g., movedto the right), the home design tool will limit the displayed searchresults to rooms having dimensions more closely matched to thedimensions entered by the user. The user can also select the amount ofrelevance the home design tool places on the openings of the room suchas any doors or windows (as indicated by slider bar 549). If the sliderbar is moved to indicate a greater emphasis on the openings present inthe room (e.g., moved to the right), the home design tool will limit thedisplayed search results to rooms having openings more closely matchedto the openings entered by the user. The user can select the amount ofrelevance the home design tool places on the products in theircollection of visual bookmarks (as indicated by slider bar 550). Thisallows the user to limit the search results to designs with similarproducts as those that have been gathered and selected by the user. Ifthe slider bar is moved to indicate a greater emphasis on the matchingthe types of products (e.g., moved to the right), the home design toolwill limit the displayed search results to rooms that include homedesign products more closely matched to the home design products storedin the user's collection of visual bookmarks. It is believed thatallowing the user to limit the displayed search results based on thetypes of products they have stored in their visual bookmarks can providethe advantage of allowing the user to more easily locate designs thatmay be of interest to him/her. For example, if the user has generatedvisual bookmarks for products with a very contemporary style, they arenot likely to have interest in a rustic, country themed design. As such,by filtering the results based on the visual bookmarks of the user, theuser may be presented with results that have a more contemporary feel.

The room dimensions, room openings, and type of home design products canbe used to filter the search results (e.g., as indicated by slider bars548, 549, and 550 respectively). Other characteristics could be used inaddition or instead of these features.

In some implementations, e.g., as shown in FIG. 28, the user can refinethe search results to display rooms with particular characteristics. Forexample, a user might desire to have a contemporary feel to the room. Inorder to filter the results to show rooms having the desiredcontemporary feel, the home design tool presents the various stylecategories (as indicated by arrow 552) and the user can select to viewonly rooms that have a desired style. In addition, the user of the homedesign tool could also select to limit the search results to roomshaving particular type tags (as indicated by arrow 553) or product tags(as indicated by arrow 554) included as part of the design. The typetags (as indicated by arrow 553) provide categories of designs that aresorted based on the type of room. The product tags (as indicated byarrow 554) provide categories of designs that are sorted based onparticular products. For example, if the user has selected a type ofcabinets they would like to use in their design, then they can searchfor only rooms that include that type of cabinets.

After the user has entered search specifications to search for roomsbased on the entered information, the home design tool presents thesearch results in a list of search results 560. The list of searchresults can be displayed in an order of relevance. For example, the homedesign tool can rank each of the search results using a relevancy scorethat is based on the filtering characteristics entered by the user. Thedesigns can be displayed in the search results list based on therelevancy score. For example, designs that have the highest relevancyscore (e.g., the designs that most closely match the enteredcharacteristics) can then be presented at the top of the search resultslist 560 so that the user can view the most relevant results first. Inthe exemplary search and search results shown in FI 28, search resultsshown in FIG. 2, the search parameters entered by the user has resultedin 5,213 potentially matching room designs. The designs have been sortedfor relevance based on the preferences entered by the user and the tendesigns that have the highest relevancy score have been displayed on theuser interface (note only 4 of the 10 are shown in FIG. 28). The usercan view additional results by selecting a button to navigate to thenext set of results ranked in terms of relevance (e.g., the resultsranked 11 to 20).

The search results can be displayed to include various information aboutthe room design. For example, as shown in FIG. 28, the search resultscan be displayed to include information that is believed to bebeneficial to the user in determining whether to view the design. Forexample, the search results can include a CAD model of the room, a shortdescription of the design, and pictures of home design products used inthe room. The search results can also be displayed to show the style,size and dimensions of the room.

Referring back to FIG. 26, after the home design tool receivesinformation about the floor plan (382) and extracts the floor plancharacteristics (384), the home design tool receives information aboutthe room type from the user (386). By providing the room type, thesearch results can be limited to the same type of room (e.g., kitchen,living room, bedroom, study, play room, dining room, etc.). The homedesign tool searches for rooms having the same room type and similarfloor plan characteristics (388). The user may also type words such asroom names, architectural styles, room characteristics and any otherterm that may help refine the search into the search field to finddesigns of interest. The system uses taxonomy based categories andfolksonomy based tags to identify matches. The system also presents theuser with a list of categories and tags to further refine the search.This search can be a vector based search in which the system identifyingsimilar designs as outlined above. The input could also be rastergraphics from a user scanned and imported sketch of their design. Aftergenerating a list of related room designs based on the search algorithm,the home design tool presents the search results to the user (390).

In some implementations, a user may input a floor plan that includesmultiple rooms into the home design tool and subsequently desire to makechanges to a particular one of the rooms. In order to make changes toone of the rooms in the floor plan it can be beneficial to have the roombe able to exist as a reference to an overall floor plan. As areferenced room that may be able to be viewed separately from the otherrooms in the floor plan, the user can view, edit, and share the roomusing the home design tool. The user can also use the home design toolto search for other rooms having similar layouts.

FIG. 30 shows a process 450 for separating or referencing a room fromadjoining rooms (e.g., rooms that share walls) in a floor plan. The usercan draw an entire floor plan using walls, doors, windows and otherobjects. In order to view a particular room from the floor plan, theuser selects a room to view. For example, the user can select the roomby clicking on the room using a mouse or other selection device. Whenthe user selects a room, the home design tool receives the selection(452) and automatically determines which walls in the floor plan thatincludes multiple adjoined rooms structurally define the selected room(454). The system determines which walls belong to the room by lookingfor walls that define the space as well as room divisions where the userhas used a room divider tool designated by a dashed line that shows thateven though two rooms are connected in an open floor plan, there are twodistinct spaces. As the user creates rooms with wall tools and roomdivider tools, the system prompts the user to name rooms. The system mayautomatically prompt the user for a name if objects related to aspecific room such as a kitchen are included in the design. The homedesign tool also automatically determines which, if any, home designproducts are included in the floor plan for the selected room (456). Thesystem determines which home design products are to be associated withthe room by the boundaries of the room and any objects such as doors,windows and wall openings are included in the boundary walls. Objectsinside the room or attached to the interior of the walls are alsoincluded as part of the room. After determining which walls and homedesign products should be included in the room, the home design toolcreates a reference of the selected room from the other rooms in thefloor plan (458) and displays only the selected room to the user (460).

FIGS. 31 and 32 show an example of referencing a single room from otherrooms in a floor plan. As shown in FIG. 31 a floor plan 410 includesmultiple rooms 412, 414, 416, 418, 420, 422 and 424 that share commonwalls. A user may desire to view and/or edit a single room from thefloor plan. For example, if the user desires to view and/or edit thedining room 414, the user can select the dining room 414 by clicking onthe dining room 414 using a computer based selection tool such as amouse or keyboard. Upon receiving the selection, the home design toolautomatically determines that the dining room 414 is defined by walls428, 430, 432, and 434 and that the dining room includes a table 438, afireplace 436 and a cabinet 440. The home design tool either generates adining room separate from the floor plan 410 based on the walls and homedesign products included in the dining room 414 (e.g., as shown in FIG.32) or simply references these elements in a separate view. As can beseen in FIG. 32, the separated dining room 414 includes the same walls428, 430, 432, and 434 and the same table 438, fireplace 436 and cabinet440 that were included in the entire floor plan 410.

After the user has referenced or separated one room from the otheradjoining rooms in a floor plan, the user can edit or change the floorplan for the newly non-adjoined room. The user may edit anything withinthe room while viewing the room independently except the walls thatdefine boundary of the room and windows, doors and wall openingsincluded in these walls. Walls, windows, doors, and door openings may beadded within the room during this view mode. After completing the edits,the user may return to view the entire floor plan and the edits appearwithin this view as well.

It is believed that referencing and/or separating a floor plan intoindividual rooms can provide various advantages. For example, bydividing floor plans into rooms within the database included in the homedesign tool, not only the entire floor plan but now every room generatedby each user can be easily explored, searched on, viewed, edited andincorporated into other designs. This allows other users to drag anentire room form another user into their floor plans as a starting pointfor their own designs. As soon as the user makes any edits to the roomdesign that they incorporated into her design, the system breaks thereference to the original design. In other implementations, the systemwill break the reference as soon as a user drags another user's roomdesign into his/her floor plan. However, users may be able to select ahyperlink back to the original design to view the original design.

In some implementations, a user may desire to combine multiple differentrooms from different users or from different floor plans to generate afloor plan that includes multiple adjoined rooms. In order to combinemultiple rooms into a unified floor plan, the home design tool includesa system that allows a user to snap together rooms that were submittedfrom two distinct users (or from the same user) and from two distinctfloor plans. A user that already has a room design in the floor plandrawing area can drag a new room into her design. The walls that areparallel to each other will merge and the system will prompt the user tochoose which wall openings they wish to delete. Until two rooms aremerged, the user can select and move an entire room by clicking on thefloor and moving it. For example, as shown in FIGS. 33 and 34, a usermight desire to combine a dining room 470 with a kitchen 472. In orderto join the two rooms (e.g., the dining room 470 with kitchen 472), thehome design tool must determine which walls need to be merged ad and howto merge the walls. For example, the dining room 470 includes a door 476between the portions 474 and 478 of one of the walls but the kitchen 472does not include a door on the wall 480 that is to be joined with thewall of the dining room 470. As such, the home design tool determineswhether to keep the door 476 when merging the two walls (e.g., as shownin FIG. 34).

FIG. 35 shows a process 480 implemented by the home design tool formerging two walls to join two rooms in a floor plan. The home designtool receives an indication from a user to merge two rooms (482). Forexample, in one implementation, a user can drag one rooms towardsanother room on a user interface to indicate that the two rooms shouldbe joined in a floor plan. When the home design tool receives anindication to join two rooms, the home design tool determines whichwalls from the two rooms are to be combined into one shared wall (484).For example, in one implementation, the system can determine that wallsthat are approximately parallel in the two rooms should be merged whenthe rooms are joined.

The home design tool determines whether either of the walls that are tobe merged when joining the rooms does not have any wall objects such aswindows, doors, or other openings (486). If one of the walls does nothave any wall objects, the system determines that the other wall (e.g.,the wall with wall objects) should be given priority when merging thetwo walls. Since the wall with wall objects (if any) is given priority,the home design tool merges the walls by adding any wall objects (ifany) from the wall with wall objects to the wall without wall objects(494). On the other hand, the home design tool determines that both ofthe walls have wall objects, the home design tool presents a wallpreference question to the user (488). In response to the wallpreference question, the home design tool receives a user selection of awall to be given preference in the wall merging process when joining thetwo rooms (490). Once the wall merging preference is known, the homedesign tool merges the walls by giving priority to the selected wallsuch that wall objects such as doors and windows on the non-selectedwall are removed and any wall objects from the selected wall become partof the merged wall in the joined rooms (492).

While in the implementation described above when both walls from tworooms to be joined include wall objects the user selects the wall to begiven priority, however other selection methods are possible. Forexample, the home design tool could assign priority to the room thatremains stationary or to the room that was inserted into the floor planfirst.

If the walls of the joining walls are different lengths, the systemidentifies how the user drags the room next to the other room todetermine the merging process. For example, if the corner (vertex) ofthe wall from the new room and the wall from the existing room arewithin two inches of each other and the walls are parallel to eachother, the system will automatically assume the user intended to mergethe two corners of the wall and position the new room accordinglywithout adjusting its size. If the vertices are not within two inches ofeach other, the system will simply assume that the user wishes to mergethe walls and not the vertices. When the user first drags a new roomtowards another room, they hold the left mouse button down. If theyhover a wall from the room they are moving over the wall of a room theywish merge with the system will allow the user to slide the new roomalong the wall of the existing room. When the left mouse button isreleased the room position is set. Dimensions may appear during thisprocess to show the offset from the corner of the wall for one room tothe corner of the wall for another room. The user can use thesedimensions to set the room position. If the user is within two inches ofthe vertex of the existing room's wall, the new room will slide intoplace to merge the vertices. A possible feature is to automaticallyadjust a wall length to of one of the joining walls when the mergeoccurs. However, this is not one of the most common approaches.

FIGS. 36A-36E show an exemplary use of the joining of distinct rooms togenerate a floor plan that includes multiple adjoined rooms. As shown inFIG. 36A, a user has generated a CAD model for a kitchen. The user canselect living room as the room type to search for a living room to addto the lay out by using the sketch pad to enter the general shape of theroom (e.g., as described above). In response, the home design toolpresents the user with various previously created rooms matching theuser's criteria (e.g., as shown below the sketchpad). In order to joinone of the living rooms presented in the search results, the userselects the design using a mouse or other input tool and drags the roomto the desired location in the layout (e.g., as shown in FIG. 36B). Whenthe user drags the rooms together, the home design tool determines howto merge the walls of the rooms to generate adjoined rooms (e.g., asshown in FIG. 36C). The user can then search for additional rooms (e.g.,a sunroom as shown in FIG. 36D) and add additional rooms to the layout(e.g., as shown in FIG. 36E).

While in the implementations described above, one wall was givenpriority over another wall when two walls are merged to join rooms in afloor plan; however other merging processes are possible. For example, auser could select particular wall objects from one or both of the wallsto remain in the joined wall. In another implementation, the home designtool could give preference to different types of wall objects and mergethe walls according to an assigned preference. For example, the homedesign tool could give priority to doors over windows or existing roomover a new room. Other priorities will be set by which types of objectsare included in the design. For example a kitchen design may be givenpriority over a living room design if the kitchen has objects like wallcabinets included on the merged wall of the design and the living roomdesign has door opening in the same location.

In an implementation, search results or suggestions may be generated andpresented to a user. The results presented to a user may include, forexample, various types of design layouts, design elements, productimages, visual bookmarks of products, or sets or groups of these. In animplementation, a process for generating and presenting search resultsor suggestions to a user may include building the results, selecting theresults, or presenting the results according to a particular priority orordering scheme, or any combination of building, selecting, andpresenting. Results may also be generated and presented to a userindependently of a search. Particular search results or suggestionspresented to a user may include built results or suggestions togetherwith selected results or suggestions. Any numbers used with regard to,for example, building, selecting, and presenting are intended to serveas non-limiting examples, and other numbers may be used.

In an implementation, results or suggestions such as layouts, productimages, or elements (or sets of these), may be built according to, orbased on any of a variety of criteria or methodologies, such as designrules; adaptive learning using neural networks that analyze prior userdesign behavior and selections; keyword associations; input receivedfrom a user; sponsorship by an advertiser or system partner; or elementsor images previously existing in, or previously selected by a user for,a layout (or set of layouts) or a set of product images (or sets ofproduct images). Results may also be built using subsets or minisets oflayouts, product images, or elements. These criteria are not exhaustiveand other suitable criteria for selection may be used.

In an implementation, results or suggestions such as layouts, productimages, or elements (or sets of these), may be selected frompre-existing or simultaneously built designs, including layouts, productimages, or elements (or sets of these) that may have already been builtby the system, used, built, or saved by a user or users, or used, built,or saved by or for sponsors of products, elements, or layouts within thedesigns, or as combinations of these. The results may be selected bydrawing from pre-existing designs or by matching up to pre-existingdesigns. The selection of results or suggestions may include selectionof results built simultaneously with the selection. The results may beselected based on any of a variety of criteria or methodologies, such asdesign rules; level of compliance with design rules; sponsorship by anadvertiser or a system partner; keyword associations; input receivedfrom a user; popularity of pre-existing designs; degree of use ofpre-existing designs; how recently a pre-existing design was used, builtor saved; adaptive learning using neural networks that analyze prioruser design behavior and selections; architecture, interior design orother design professional associations; or elements or images previouslyexisting in, or previously selected by a user for, a layout (or set oflayouts) or a set of product images (or sets of product images). Resultsmay also selected by drawing from, or matching up to, pre-existingsubsets or minisets of layouts, product images, or elements. Thesecriteria are not exhaustive and other suitable criteria for selectionmay be used.

In an implementation, results or suggestions such as layouts, productimages, or elements (or sets of these), may be presented according to aparticular priority or ordering scheme. The results may be presented inorder of any of a variety of criteria or methodologies, includingpopularity; degree of use; how recently used; or elements or imagespreviously existing in, or previously selected by a user for, a layout(or set of layouts) or a set of product images (or sets of productimages), for example. In an implementation, results built by the systemmay be presented according to any of the criteria used in building theresults, and results selected by the system may be presented accordingto any of the criteria used in selecting the results.

In an implementation, design rules may include interior or architecturaldesign guidelines, recommendations, standards, norms, methodologies,philosophies, or styles, individually or in combination, implemented asdesign rules in, for example, algorithms of the system. The design rulesmay be updated from time to time to reflect changes or advancements in,for example, interior and architectural styles. While algorithms may bebased on design rules, in some implementations the design rules may beguidelines or considerations rather than strict rules.

Search results or suggestions may be filtered to avoid or reduceredundancy prior to or in conjunction with presentation to a user. In animplementation, redundant individual results may be moved from groups ofresults prior to or in conjunction with presentation to a user. Forexample, a first group of results may be selected for presentation to auser. In an implementation, prior to or in conjunction with presentationto a user, any redundant results from the first group may be removedfrom the first group and removed altogether. In another implementation,prior to or in conjunction with presentation to a user, any redundantresults from the first group may be removed from the first group andplaced into a second, or another, group of results. Other results may bebrought into the first group to replace any removed redundant results,so that the first group ultimately presented to a user contains the samenumber of results before and after removal of the redundant results.Other techniques that may avoid or reduce redundancy in search resultsor suggestions made to a user may be used.

Although several examples are shown using a kitchen and associatedlayouts, elements and products, many other room type driven layouts anddesigns are possible, including for room types such as dining rooms,bathrooms, living rooms, family rooms, entertainment rooms, dens, andbedrooms, for example. The systems and methods described may be appliedto these other room types, and associated layouts, elements, andproducts.

The term “set of products” or “combination of products” may be usedinterchangeably with the form in which the products are presented, suchas “set of product images.” Likewise, the term “product” may generallybe used as shorthand for a home design product, or a landscape designproduct. Generally, a product image may be associated with acorresponding 2D drawing or a 3D model of the product showing in theimage, or both. The product image may have associated tags for theproduct shown in the image. The tags may include certain key or corecharacteristics for that particular product, such as sizing information.The characteristics may have been scraped from a manufacturer's websiteeither automatically, or in response to a request made by a user of thesystem. The product sets or product images shown may be integrateddirectly with a retailer's catalog of products, for example, so that thecontent and core characteristics may be provided directly from theexternal retailer's website.

Although search dialogue boxes or text boxes may not be illustrated oneach webpage of the home design tool, in an implementation, the user isprovided with the opportunity to refine the searching at any point, orat least at numerous points, of the design process.

FIG. 37 shows a screenshot of a webpage 700 associated with animplementation of the home design tool that shows a design searchinterface where a user may search for, and select from, pre-existing orsystem-built designs.

FIG. 38 shows a screenshot of a webpage 750 associated with animplementation of the home design tool that shows a product set searchinterface where a user may search for, and select from, an array ofproduct sets.

In FIG. 37, a user may search for designs from other users, for example,and may refine her design search in several ways. First the user mayrefine the room type as, e.g., a “Kitchen” at box 702. A user may alsosketch a structural design layout 704 of a room in a sketchpad 706 anddefine the shape and features of the room by using structural elementssuch as a window 708 or a door opening 710. In an implementation, asketch drawn and entered by the user may be used to drive search resultsand suggestions of collected designs being presented to the user.

In FIG. 37 (and FIG. 38), a user may also drag and drop home designproduct images such as an image of a stove 714 (752 in FIG. 38) fromsearch results 712 (754) into a refine search box 716 (756). Once aproduct or products is moved into the refine search box 716 (756),search results may be returned that include results with the product orproducts included. All of the products returned in various product setsmay or may not be an exact match of the product or products moved intothe refine search box 716 (756), although in this instance, the sets inthe search results 712 (754) shown in FIG. 37 (FIG. 38) all include thestove 714 (752). Products sharing similar characteristics with theproduct or products may be included in the search results 712 (754), forexample, an oven that is stainless steel like the stove 714 (752).

As shown in FIG. 37, a user may enter search terms into a text box 724near the top of the webpage 700 and may select “Search Designs.” In theexample shown in FIG. 37, the user entered “Sun filled kitchen, gerefrigerator” and received 10,253 design matches with the first (say)twenty collected designs being shown to the user. The system mayseparate words or combine words and may compare the received words to adatabase of design keywords. The word “kitchen” may tell the system thatthe user wants the room type to be a kitchen. “Sun-filled” may indicatetwo things. First, it may cause the system to search for structuralelements such as windows or doors that have glass in them, or dooropenings that open into another room, and thus will allow a lot of lightinto the room. Second, “Sun-filled” might also indicate to the systemthat the user wants lighter colors in the design. The system mayrecognize the word “ge” as indicating a brand of products and may assignthose products a higher priority in returned search results. The wordcombination “ge refrigerator” would indicate a specific type of productand a specific brand that may likewise be assigned a higher priority inreturned search results.

Generally, keywords may help the system better target search results andsuggestions to the user. The system may examine entered words forkeywords indicating, for example, (1) the structure of a room or thetype of room (e.g., “kitchen”); (2) types of product layouts for a room(e.g., “u-shaped kitchen cabinets,” or “l-shaped kitchen cabinets,” or“kitchen cabinets with an island”); (3) sets or combination of products(e.g., “sun-filled” may indicate a lighter colored kitchen or brighterroom); (4) individual products within the design (e.g., refrigerator);and (5) a match with descriptive text and associated tags about theactual design.

Tags associated with product images or products or with a design mayinclude any information to help identify the product, such asdimensional, color and other information that may have been scraped fromthe product manufacturer's website, or provided by the manufacturer, aswell as user or defined tags that describe the product or that describea user's design. User defined tags may be assigned higher prioritywithin the system for search results.

As shown in FIG. 37, the search results 712 returned to a user mayinclude product layouts 718 a, 718 b, 3D design layouts 720 a, 720 b,and sets of product images 722 a, 722 b as part of separate collecteddesigns 724 a. The system search engine may employ redundancy avoidanceor minimization techniques, as described herein, to present more uniqueand potentially more helpful results to the user. The search results 712include collected designs 724 a, 724 b from particular users of thesystem; however, in other implementations, the design search interfacemay dynamically build design results or suggestions for presentation toa user, rather than drawing on pre-existing designs. FIG. 37 shows afirst design 724 a by an interior design student Jane Doe who hasprovided a description of her design “Traditional kitchen designoffering sunlight” (thus returned as a result to the text search “Sunfilled kitchen, ge refrigerator”). A second design 724 b is authored byBob Smith, a contractor who likewise provided a design description forhis collected design that also clearly relates to the search termsentered by the user.

As described above, the system may rank designs according to any numberof factors. In an implementation, the system takes into account theprofession, or occupation, of a users when selecting from pre-existinguser designs and prioritizing them for presentation to a user. In animplementation, there may be a hierarchy of professions. An architectmay be ranked ahead of an interior designer for some situations, andvice versa. A contractor, may, in turn, be ranked below both an interiordesigner and an architect, and an interior design student may be rankedbelow all three for purposes of selecting and prioritization of results,for example.

In an implementation, searches from the design search interface may beranked or ordered according to popularity (“Most Popular”) or howrecently used (“Most Recent”). A conflict in popularity, for example,(two potential results having the same popularity) may be resolved byresort to the criteria of the profession of the designer in theprofessional hierarchy that may be programmed into the system.

Users may also use the design search interface shown in FIG. 37 or theproduct set search interface in FIG. 38 to bypass various steps in thesystem. For example, a user could select a certain set of products suchas set of products 722 a in FIG. 37 or a set of products 758 b in FIG.38 by selecting an option to use the set of products to create a design.A design wizard may be automatically launched that may prompt the userto choose a room layout in which to create her design. In this way, auser may move more quickly to a 3D design layout that includes theselected product set 722 a or the product set 758 b.

A user may also, for example, use the design search interface to bypassthe structural design and product layout portions of the system. A usermay select the product layout 718 b by selecting an option to use theproduct layout to create a design. A design wizard may be automaticallylaunched that may prompt the user to choose from an array of sets ofproducts to insert into the product layout 718 b to create more quicklya 3D design layout based on the product layout 718 b.

Referring again to FIG. 38, the product set search interface mayillustrate some product set search capabilities of the home design tool.Product set searches may occur in a variety of ways. In animplementation, product set search results 754 may be returned thatdisplay product content from 3D designs that other users saved in thesystem. These pre-existing entire product sets may show all of theproducts that a user used in a particular 3D design layout. For example,for product set or set of product images 758 a a user may have saved a3D design layout that included a refrigerator, a light fixture, a sink,a stove, a kitchen cabinet face and various flooring or wall colors andtextures, all shown in product set 758 a.

In an implementation, the system may dynamically build product sets fromindividual products or minisets of products based on, for example,products either selected by a user in a previous screen or product(s)that were dragged and dropped by a user into the refine search box 756.

Minisets of products generally may be combinations of one or moreproducts that are grouped together by the system, users of the system,or advertisers or partner external sites. The minisets will generally besmaller than an entire set of products and may be used by the system todynamically build an entire set of products or a larger miniset ofproducts that includes the miniset, or to influence the building ofanother miniset or entire set of products. An entire set of products maybe generally a set of products that fully defines a 3D design layout fora particular room, or simply a larger set of products that either morefully defines a 3D design layout than the minisets used to build theset, or that is simply larger than the minisets.

In FIG. 38, product set search results or suggestions may be presentedto a user in order of, for example, popularity (“Most Popular”), or howrecently used (“Most Recent”). For example, six people may have used theproduct set 758 a in their 3D design layouts, making that product setthe most popular for this particular search. The system search enginefor product set search may employ redundancy avoidance or minimizationtechniques, as described herein, to present more unique and potentiallymore helpful product set search results or suggestions to the user. Thewebpage 750 shows a text box 760 where a user may enter text basedsearch terms (here, “Kitchen design with GE refrigerator”) and therefine search box 756 where a user may drag and drop products or sets ofproducts to refine search results. The user would select “SearchProducts Sets” for a text based search and the words entered would beanalyzed in similar fashion to that described with reference to FIG. 37,in this case yielding “Kitchen” product sets with “ge refrigerators,”for example.

The system, in addition to using descriptive keywords that may have beenused by a user to describe her designs (and thus the product setsassociated with her designs) to aid users in product set searches, mayalso consider how such descriptive keywords correlate with selections ofthe product sets by other users in the system. For example, the user mayalso have been prompted to assign descriptive keywords to a 3D designlayout (that included the products that make up the product set), suchas (for style) “Contemporary” or “Modern” or “Traditional” or Historic,”and one or more of these descriptive terms may have been applied to theproduct set. Over time, if these product sets are selected by users whoseek that type of design style, for example, the system may learn thatthe product set is truly “Traditional” or the like.

As described above, in FIG. 38, a user may also drag and drop homedesign product images such as an image of a stove 752 from searchresults 754 into a refine search box 756. Once a product or products(e.g., minisets of products) is moved into the refine search box 756,search results may be returned that include results with the product orproducts included. All of the products returned in various product setsmay or may not be an exact match of the product or products moved intothe refine search box 756, although in this instance, the sets in thesearch results 754 shown in FIG. 38 all include the stove 752. Productssharing similar characteristics with the product or products, such assimilar finish options, similar look, or similar colors, may be includedin the search results 754, for example, an oven that is stainless steellike the stove 752.

In an implementation, these returned product sets may include productsets saved by other users, but product sets may also be dynamicallybuilt from the products moved into the refine search box 756. If a user,for example, drags and drops a range (such as range 752), a refrigeratorand possibly a dishwasher (themselves all from different productminisets) into the refine search box 756, these products may all bestainless steel and may be from certain brand name manufacturers.However the user, by this combination, has not given the system muchinformation as to her design tastes or inclinations. If the user drag akitchen cabinet face into the refine search box 756, then the systembegins to learn more information about this user's design interestsbecause such a selection may be more specifically correlated to, forexample, what kind of style flooring, countertops, paint color forwindows, for doors, that the user may want to select. Thus, minisets ofproducts may allow the system to refine with product sets anddynamically built product sets that may be more tailored to the user'sinterests than pre-existing sets of products stored by the system.Because the user in the example above chose an instance of a kitchencabinet, or chose a cabinet and a flooring option, the system mayintelligently examine the designs of other users and the productcombinations used by these users in minisets, to define what otherproducts might make sense in an overall product set—product sets thatthe system may correlate should work well together.

The right hand side of the webpage 750 of FIG. 38 shows a product set762 offered by a sponsor or advertiser. A picture 764 (or 3D layout (notshown)) of an example kitchen that incorporates the sponsor's productset 762 is also shown. A sponsor may want to offer an individual product(as part of an entire set of products), miniset of products, or even anentire set of products. Advertisers may bid on minisets of products. Forexample, an advertiser may bid on a combination of a refrigerator, astove and a dishwasher. The advertisers may bid against one another forminisets on a cost per click basis. Advertisers may bid on items such askeywords (e.g., “ge refrigerator”), product sets or minisets, orindividual product types to meet their particular sales objectives. Inthe product set 762, for example, the sponsor might have been thehighest bidder for the stove 752.

As described above, the product images making up a set of products mayhave associated tags for the product shown in the image. The tags mayinclude certain key or core characteristics for that particular product,such as sizing information. The characteristics may have been scrapedfrom a manufacturer's website either automatically, or in response to arequest made by a user of the system. In an implementation, the productsets or product images shown may be integrated directly with aretailer's catalog of products, using XML (Extensible Markup Language)for example, so that the content and core characteristics may beprovided directly from the external retailer's website. In animplementation, a product set search interface similar to that shown onthe webpage 760 may also be part of an external partner's websiteinterface (possibly as part of a larger design tool) with, for example,the system's engine operating in the background so that a user may seeonly the external partner's interface. In an implementation, integratingthe system product set search capabilities (alone or as part of anoverarching design tool) with a partner's retail catalog of products mayallow the system to provide the partner with data about things such asproduct sets, product combinations, or design layouts from users of theintegrated product search system. For example, the system may gatherinformation about how users combine products and choose to apply thoseproducts to design layouts that may be valuable to external partners.

FIGS. 39-47 show screenshots of webpages following a user's progressionthrough a design process. The design process takes the user in broadterms from expressing design goals (FIGS. 39 and 40); to being presentedwith product layout results or suggestions and choosing a product layout(FIGS. 41 and 42); to being presented with product set results orsuggestions and choosing a product set to apply to the product layout(FIGS. 43 and 44); and to viewing a 3D design layout with the productsets included and being presented with complementary product results orsuggestions to choose from to add to the 3D design layout (FIGS. 45-47).

In implementations of the design process shown in FIGS. 39-47, atvarious stages the system search engine may employ redundancy avoidanceor minimization techniques, as described herein, to present more uniqueand potentially more helpful results to the user. The search results orsuggestions may include layouts, elements, or products, from particularusers of the system; however, in other implementations, the designsearch interface may dynamically build design results or suggestions(based on, for example, design rules) for presentation to a user, ratherthan drawing on pre-existing designs. A variety of criteria may beconsidered in building, selecting or presenting results or suggestionsin a particular order.

FIGS. 39 and 40 show screenshots of webpages 770, 780 that have textboxes 772, 782 where a user may enter design search terms that may beindicative of the user's goals for a particular design. A user may haveselected an option to “start a new project” (not shown) to have beenprovided with the webpages 770, 780. The user may click on “StartWizard” to begin the design process. As described above with referenceto the system processing words from the text box 724 of the designsearch interface of FIG. 37, in an implementation, the system mayseparate words or combine words and may compare the received words to adatabase of design keywords. Generally, keywords may help the systembetter target search results and suggestions to the user. The system mayexamine entered words for keywords indicating, for example, (1) thestructure of a room or the type of room (e.g., “kitchen,” or where“open-floor plan” (see FIG. 40) may indicate wide door openings); (2)types of product layouts for a room (e.g., “u-shaped kitchen cabinets,”or “l-shaped kitchen cabinets,” or “kitchen cabinets with an island”);(3) sets or combination of products (e.g., “sun-filled” may indicate alighter colored kitchen or brighter room); (4) individual productswithin the design (e.g., refrigerator); and (5) a match with descriptivetext and associated tags about the actual design.

In an implementation, the system may employ algorithms that may beperformed in response to search terms entered that may cause the systemto target search results and suggestions to the user throughout severalsteps of a design process. That is, a user who enters very specificdesign search terms may be taken toward a final design in several steps,rather than be provided with a very specific set of layouts or productsets to choose from. In an implementation, the entered search terms maydefine what happens at various stages of the design process, not justimmediately.

For example, if in the design goals, the user had used the word“beachhouse,” the home design tool may not necessarily present the userwith only “beachhouses.” Rather, the user may still be prompted todefine structures such as rooms, but if the design progressed to a stageat which the user defined a house (e.g., in an exterior model), theentered words (including “beachhouse”) may guide the user and the searchresults and suggestions presented to the user in that direction. Thatis, the choices that would be suggested to a user may be guided by thewords that the user initially entered several pages back in “DesignGoals.”

FIG. 41 shows a screenshot of a webpage 800 provided to a userfollowing, for example, entry of the user's design goals at either,e.g., FIG. 39 or 40. In FIG. 41, results or suggestions of productlayouts, that is, structural design layouts that have been populatedwith elements such as product elements, are presented to a user basedon, and in order of, popularity, or how recently used, for example. Thesearch results may include product layouts from particular users of thesystem; however, in other implementations, the design search interfacemay dynamically build design results or suggestions (based on, forexample, design rules) for presentation to a user, rather than drawingon pre-existing designs. A variety of criteria may be considered inbuilding, selecting or presenting results or suggestions in a particularorder. For example, the product layout presented may have been built orselected based in part on input from the user in a prior screen, such asthe design goals webpages 770, 780.

FIG. 41 provides a “Start from Scratch” option 802 to begin the designat a more basic structural level (see discussion of figures beginning atFIG. 48), as well as an “Import 2D drawing” option 804. A user may begiven the option of importing a JPEG image, GIF file, or other graphicfile that shows a room design or other structural layout, and the usermay be allowed to draw or sketch room layout on top of the 2D drawing tobegin the design process.

In FIG. 41, beginning the “Step 1: Choose a Layout Option” portion of animplementation of a design process, a set of kitchen product layouts 806and a set of bathroom layouts 808 may be presented to the user. The usermay choose which room to focus on by selecting either of these options,and here may select “Kitchen Layouts,” sending the user to the webpage810 in FIG. 42.

In FIG. 42, a screenshot of the webpage 810 shows a product layoutsearch interface portion of an implementation of the design process forFIGS. 39-47, where the user may search for, and select from, sets of 2Ddrawings of product layouts.

From the selection made on the previous screen, the room type is“Kitchen” at box 812. The user may have several options at the webpage810. A user may sketch a structural design layout 814 of a room in asketchpad 816 and may define the shape and features of the room by usingstructural elements such as a window 818 or a door opening 820. In animplementation, a sketch drawn and entered by the user may be used todrive search results and suggestions of 2D drawings of product layoutsbeing presented to the user. In an implementation, if a user sketches astructural design layout 814, the system will return the most popularproduct design layouts that have designs that match, or are very similarto, the requirements of the structural design layout 814. The system maydraw on preexisting user or sponsor product layouts or may dynamicallybuild product layouts that meet, or come close to meeting, thestructural design layout requirements made by the user in the sketch814. The user may, using the sliding scales corresponding to “Shape”822, “Outside Dimensions” 824, and “Structural Elements” 826, affect thesearch results by defining the relative importance of the threecriteria, shape, outside dimensions and structural elements. Forexample, while the search algorithm may emphasize the shape of the roomabove the other criteria, by moving the Structural Elements” scale 826to “+”, the user may cause the system to provide search results thatmatch, or come close to matching, the structural elements 818, 820within the sketch/structural design layout 814. In an implementation, bydefault, the system may return a group of the most popular results 828that match or are close to matching the sketch 814, but may also presentthe results in order of the most recently used. The system may alsopresent sponsored product layouts 830 that match, or come close tomatching, the sketched layout. In an implementation, the system may havethe capability to dynamically change product layouts (sponsored or userdesigns) to reflect dimensional changes made by the user to the room 814in the sketchpad 816.

If a user does not use the sketchpad 816, the “Most Important Aspects”search criteria 822, 824, 826 may be grayed out and the user may bepresented with a group of product layouts 828. In an implementation, bydefault, the suggestions for product layouts may be drawn from the mostpopular product layouts, and the returned group of product layouts maybe ordered according to popularity. In other implementations, the systemmay select and order product layouts based on criteria such as howrecently used. Other selection criteria may be used instead of or inaddition to these criteria.

In an implementation product layout results or suggestions may also bedynamically built by the system to, e.g., reflect user input such asthat entered in the “Design Goals” text boxes 772, 774 of FIGS. 39 and40, or to, for example, comply with design rules implemented in thesystem.

To generate the search results as shown in FIG. 42, the system searchengine may employ redundancy avoidance or minimization techniques, asdescribed herein, to present more unique and potentially more helpfulproduct layout results to the user. Specifically, search results orsuggestions may be filtered to avoid or reduce redundancy prior to or inconjunction with presentation to a user. In an implementation, redundantproduct layouts may be moved from groups of results prior topresentation to a user. For example, a first group of (say) twentyproduct layouts results may be selected for presentation to a user basedon popularity. In an implementation, prior to presentation to a user,any redundant product layout results (e.g., the less popular redundantresults) from the first group may be removed from the first group oftwenty and removed altogether. In another implementation, prior topresentation to a user, any redundant results (e.g., the less popularredundant results) from the first group of twenty may be removed fromthe first group and placed into a second, or another, group of results.Other results may be brought in to the first group of twenty to replaceany removed redundant results so that the first group ultimatelypresented to a user (e.g., group of product layouts 828 in FIG. 42)contains the same number of results before and after removal of theredundant results. Other techniques that may avoid or reduce redundancyin search results or suggestions made to a user may be used.

In moving redundancies from a group of results, the emphasis on whatredundancies to consider may vary depending on the particular part ofthe design process. For product layout suggestions, for example, theemphasis may be more on avoiding or limiting redundancies in productlayout characteristics, so that a user may be presented with differentdesign options such as differently shaped kitchen cabinet arrangementsor sinks, refrigerators or ranges being placed in different parts of thekitchen, rather than the same basic group of designs (e.g., kitchenproduct layouts that all have “l-shaped” kitchen cabinet layouts. Forproduct layout suggestions, structural element redundancies may be lessimportant. An example of this may be seen in the group of suggestedproduct layouts 828 and the sponsored product layout 830 shown in FIG.42. Each kitchen product layout 832, 834, 836, 838, 840, 842, 830includes a window (such as window 844 in layout 832) in the samelocation, while the actual product elements within the product layoutsshow more variation.

The user may select from product layout options such as the productlayout 834 or the sponsor product layout 830, for example. Either path(choosing a product layout from results or suggestions or a sponsorproduct layout) may take the user to some version of a product setsearch webpage 850, shown in FIG. 43, thus beginning the “Step 2: ChooseProduct Set(s)” portion of an implementation of a design process.

In an implementation, if, for example, the user chooses the sponsorproduct layout 830, the user may be presented with a choice of productsets 852 a, 852 b, 854 a, 854 b all sold by the sponsor of the productlayout 830. A “Sponsor Product Sets” box on the right may not be shownsince all product sets shown on the webpage 850 may already be sponsoredby the sponsor. The product set results or suggestions may typically beproduct sets that have been pre-defined by the sponsor, but in someimplementations may be dynamically built product sets of the sponsor'sproducts based on the prior selected product layout and or may includethe most popular results based on product sets from pre-existing userdesigns that use the sponsor's product sets. The user could then selectfrom different kitchen styles such as “Traditional” 856 or “Country”858. The product set results or suggestions may generally be based on,or tied closely to the product layout 830 that the user chose in theprior webpage 810 (FIG. 42).

If the user chooses one of the non-sponsored product layouts such asproduct layout 834 in FIG. 42, the user may be likewise presented (seeFIG. 43) with groups of product set results or suggestions in differenttypes of kitchen styles (“Traditional” 856, “Country” 858). Generally,based on the product layout that the user chose in the prior screen, thereturned product set results or suggestions may be heavily tied to theproducts in the chosen product layout.

In an implementation, a product layout may include generic productelements (e.g., “refrigerator”) that may not be associated with aspecific brand of product (e.g., “name brand refrigerator”), but thatstill may present restrictions on the product sets that may be presentedto the user on the webpage 850 of FIG. 43. For example, the product setresults or suggestions may each include a refrigerator or a refrigeratorhaving dimensions that satisfy the product layout dimensions. In animplementation, a product layout may include product elements that areassociated with a specific brand of product (“name brand stove”) ormodel of a product (“name brand stove model 4540”), and may thus presentadditional restrictions on the product sets that may be presented to theuser on the webpage 850. That is, the product set results or suggestionsmay each include the specific brand product or specific brand and modelproduct.

Other products that are either not in or specified in the product layoutmay be more easily left out of the product set results or suggestions.Examples may include lighting fixtures, paint colors or other surfacetextures, or possibly a dishwasher.

Groupings of product sets presented to the user may successively be lessand less tied to the prior chosen product layout and may leave outsuccessively more products. In general, however, the product set resultsor suggestions may be closely tied to the prior chosen product layout.

A user may select one of the different kitchen styles such as“Traditional” 856 or “Country” 858 in FIG. 43. According to the exampleprogression, the user may select “Traditional” 856 and be taken to thewebpage 880 shown in FIG. 44 where “Traditional” kitchen product setresults or suggestions may be shown to the user.

FIG. 44 includes on the right hand side a product set 882 offered by asponsor (similar to product set 862 in FIG. 43. A picture 884 or otherrendering of an example kitchen that incorporates the sponsor's productset 882 is also shown. As discussed above with reference to FIG. 38, asponsor may want to offer individual products, minisets of products, orentire sets of products, and sponsors may bid against one another forgetting these products before users of the system. The system maydynamically build the product sets 862, 882 (FIGS. 43 and 44) based onthe product layout chosen by the user, although sponsor product sets mayalso be pre-defined. The sponsor product set 882 may match all productsspecified in the product layout or may include a new product that thesponsor wants to include to the set.

FIG. 44 may also permit a user to preview a 3D layout 888 byhighlighting any of the traditional kitchen product set results orsuggestions (here, product set 886 is highlighted). In animplementation, the 3D layout preview 888 may use grayscale or anothershade to indicate products that have not been filled in on the 3D layoutpreview 888.

FIGS. 43 and 44 are similar to FIG. 38 discussed above. Product setsearch results or suggestions 864, 890 may display product content from3D designs saved by other users in the system that match the productlayout. For example, for the product set or set of product images 886 a,a user may have saved a 3D design layout (matching the chosen productlayout) that included a refrigerator, a light fixture, a sink, a stove,a kitchen cabinet face, and various flooring or wall colors andtextures, all shown in product set 886 a. In an implementation, thesystem may dynamically build product sets from individual products orminisets of products, here based potentially on the chosen productlayout or, possibly, product(s) that were dragged and dropped by a userinto the refine search box 890 such as the stove 892.

As in FIG. 38, product set search results or suggestions shown in FIGS.43 and 44 may be presented in order of popularity, or how recently used.The system search engine for this product set search aspect of thedesign process may employ redundancy avoidance or minimizationtechniques, as described herein, to present more unique and potentiallymore helpful product set search results or suggestions to the user.Product set results or suggestions created by using the refine searchbox 890 may be drawn from existing product sets based on user designs ormay cause new product sets to be dynamically generated. The productimages making up a set of products may have associated, tags forproducts shown in the images. The tags may include key characteristicsthat may have been obtained through a variety of ways, as describeherein, such as automatic scraping from the product manufacturer'swebsite, through integration with an external partner's website, asdiscussed with reference to FIG. 38.

FIG. 45 shows a screenshot of a webpage 900, that shows what may be afinalization of the design process. According to the example progressionof the design process shown in FIGS. 39-47, the user may select aproduct set such as product set 886 a in FIG. 44 and be taken to thedesign interface shown in FIG. 45. The product set 886 a may beassociated with corresponding 3D models such as the 3D models 904, 906,908. The 3D model 904 may be a representation of the sink product image896 in the product set 886 a, while the other 3D models 906, 908 may berepresentations of the stove and the refrigerator product images 896,898. The selected product layout 834 (from FIG. 42) may be converted toa 3D design layout 902, shown in FIG. 45, with the 3D design layout 902incorporating 3D models (including models 904, 906, 908) associated withthe product set 886 a chosen by the user in the previous screen.

The design interface of FIG. 45 includes the 3D design layout 902 and,on the right-hand side of the screen, an interactive design palette area910. In an implementation, the 3D design layout 902 may use grayscale oranother shade to indicate products that have not been filled in on the3D design layout 902. That is, for example, the product set 886 aapplied to the product layout 834 may not have included all of theproducts that may be needed to match all of the product elements in theproduct layout 834. Or, for example, products such as paint colors forthe trim or the walls, or flooring options such as tile or wood, werenot included as part of the product set and thus do not appear in the 3Ddesign layout 902.

In an implementation, the system, via the interactive palette area 910,may allow the user to select things in the 3D design layout 902 that areshaded in grayscale (for example) and as such may need to be defined, orotherwise drag or drop product (such as finishes) from the interactivepalette area 910 onto the objects that may need to be defined. In animplementation, the system may suggest complementary products to theuser by tailoring search results or suggestions based on the context ofthe products already included in the 3D design layout 902.

In an implementation, the system may also allow the user to selectproducts that have already been filled in on the 3D design layout 902,such as the sink 904, and to get suggestions for other sink productsthat might also be used in the room.

The system may provide an interactive shopping environment for usersusing the interactive palette area 901 by utilizing search algorithmsthat (1) may draw individual products, or minisets of products, from,for example, products already used in the pre-existing designs of otherusers; or that (2) may build individual product or product minisetresults or suggestions based on criteria such as design rules, oradaptive learning using neural networks that analyze prior user designbehavior and selections, or products or structural elements alreadyincluded in the 3D design layout 902; or do both (1) and (2), forexample.

In an implementation, the system search algorithms may examine the setof products included in the 3D design layouts as individual products oras minisets and, based on the product or product(s) being examined, maycompare the user's design decisions to those of other users (forexample) to better suggest complementary products to the user. Forexample, a user may have selected a certain type of countertops and acertain type of kitchen cabinets for her 3D design layout. The systemmay leverage these product combination choices to suggest complementaryproducts based on the design choices of other users who may have madethe same selections for countertops and kitchen cabinets. Thus, thesystem may present additional product images to the user in theinteractive palette area 910 based on the 3D models (or the productsunderlying the 3D models) incorporated into the 3D design layout 902.

In the particular example shown in FIG. 45, the user may be presentedwith flooring options because the user may have not yet selected aflooring option or because the user has decided to view flooring optionsto replace an existing flooring selection. The word “Flooring” 912 is inlarger type in the palette area 910 to indicate that it has beenselected and coordinating flooring products such as granite tile 914,marble tile 916, and wide pine flooring 918 are presented as results orsuggested options.

A user may select a “Product Sets” box on the left hand side of FIG. 45to reveal a dropdown set of product sets as shown in the webpage 930 ofFIG. 46. The “Active Product Set” 932 is the set of products presentlybeing used in the 3D design layout 902, while the “Product Set 2” 934 isan alternative set of products that a user may highlight to have thesystem automatically apply to the 3D design layout 902. Two otherproduct sets, the “Suggested Product Set from Sponsors” 936 and the“Product Sets Suggestions” 938 may also be presented to the user. In animplementations, the suggested product sets 936 may change as a usermodifies the 3D design layout 902, by for example, adding, removing, orchanging products in the design.

A 3D design layout will typically include a variety of products. In animplementation, users may be prompted to save individual products,minisets of products, or entire sets of products depending on the full3D design layout, an aspect of the layout such as a particular designstyle. Users may explore design options by returning to a previouslysaved products set, modifying or editing the sets, and saving themodified sets.

In an implementation, the system may not know what to do with aparticular product selected by a user as part of a product set, and mayquery the user about the product. On webpage 950 of FIG. 47, a user maybe told that the system is unsure where a lighting fixture 952 that wasincluded as part of the user's chosen product set (shown as product set932) would be located in the 3D design layout 902. The user may beprompted or presented with a link 954 that the user may select to switchto a 2D product layout view mode (not shown in FIG. 47) where the usermay insert the lighting fixture 952 in the 2D product layout at alocation of her choice. Other situations may cause the system to ask theuser to insert a product manually into the 2D layout, for example, ifthere is conflicting information that the system may not know how tohandle.

In an implementation, a user may use the 3D design layout 902 to definea trim or a baseboard style, color or texture for the room. Such trim orbaseboard definitions may typically be used throughout an entire house,so the system may be configured to retain the trim or baseboarddefinitions and to automatically apply them to any rooms that the usermay have associated, or may subsequently associate, with the 3D designlayout 902. Similarly, certain colors or textures used in one room maytypically complement colors or textures used in another, typicallyadjoining room in a house, so that in an implementation the system maybe configured to retain colors or textures used in one room and tosuggest (or even automatically apply) complementary colors and texturesfor any rooms that the user may have associated, or may subsequentlyassociate, with the 3D design layout 902.

FIGS. 48-65 show screenshots of webpages following a user's progressionthrough another design process. The design process takes the user inbroad terms from being presented with structural elements with which todesign and modify the structural design layouts of one or more rooms(FIGS. 48-54); to being presented with exterior structural elements tomodify an exterior 3D structural design layout (FIGS. 55-57); to beingpresented with product layout results or suggestions and choosing aproduct layout (FIGS. 58-62); to being presented with product setresults or suggestions and choosing a product set to apply to theproduct layout (FIGS. 63 and 64); and to viewing a 3D design layout withthe product sets included and being presented with complementary productresults or suggestions to choose from to add to the 3D design layout(FIG. 65).

In implementations of the design process shown in FIGS. 48-65, atvarious stages the system search engine may employ redundancy avoidanceor minimization techniques, as described herein, to present more uniqueand potentially more helpful results to the user. The search results orsuggestions may include layouts, elements, or products, from particularusers of the system; however, in other implementations, the designsearch interface may dynamically build design results or suggestions(based on, for example, design rules) for presentation to a user, ratherthan drawing on pre-existing designs. A variety of criteria may beconsidered in building, selecting or presenting results or suggestionsin a particular order.

In an implementation, the design process shown in FIGS. 48-65 may assista user in learning about the overall system design interface and aboutdifferent features, options, and capabilities of the home design tool sothat a user may be better prepared to work in a 3D design layout relatedtool and to select from, e.g., presented suggestions of complementaryproducts to apply to a 3D design layout.

According to the example progression of the design process shown inFIGS. 48-65, a user may begin at “Step 1: Define the Structure” onwebpage 1000 of FIG. 48 by adding a room to form a structural designlayout 1002 of, in this instance, a kitchen. A user may drag and drop arectangle shaped room 1008 from a set of structural elements 1004 (heredifferent shaped rooms). The user may be prompted to name the room andmay be, for example, prompted to enter room dimensions. The user mayalso click in and type in dimensional values. The system mayautomatically resize the 2D drawing or structural design layout 1002according to the dimension. The user may choose to “Add another Room”1006, which moves the user to the webpage 1010 of FIG. 49 where a usermay drag and drop a rectangle shaped room 1008 from the set ofstructural elements 1004 (here different shaped rooms) to form astructural design layout 1016 that may include both a kitchen 1002 and adining room 1014. The system may automatically merge the 2D drawing ofthe new dining room 1014 with the existing 2D drawing of the kitchen. Asfor the kitchen, the user may be prompted to enter dimensions or clickand type dimensions onto the 2D drawing. The user may add another roomor proceed to the “Next Step” 1018.

The user may draw from additional sets of structural elements such aswindows 1022 (FIG. 50), doors or door openings 1032 (FIG. 51),fireplaces 1042 (FIG. 52), sets of stairs 1052 (FIG. 53), and closets orpantries 1062 (FIG. 54). The structural elements shown in these FIGS.50-54 may be dragged and dropped onto and automatically inserted intothe various structural design layouts shown in the FIGS. 50-54, andusers may add, or be prompted to add, dimensions for these structuralelements when these elements are dragged and dropped or thereafter. Inother implementations, a variety of structural elements may be availablefor selection on fewer webpages or a single webpage. The sets ofstructural elements 1022, 1032, 1042, 1052, 1062 may be presented asresults or suggestions to the user based on any of a variety ofcriteria, such as design rules, the design of other users, adaptivelearning based on neural network analysis of the design decisions ofother users and the like. The criteria may also include, for example,dimensions of the structural design layout of the rooms, such as kitchen1002 and dining room 1014; or the structural elements already selectedby the user for the design. As more structural elements are added, theresults or suggestions may be dynamically refined to reflect themodified structural design layout.

In FIGS. 50 and 51, doors or door openings may be structurally moreimportant than windows and may be added to the structural design layoutsprior to windows. The set of windows 1022 of FIG. 51 may includedifferent styles of windows (such as casement or double-hung) as well asdifferent types of windows (e.g., bow window, bay window, “standard”window). In FIG. 52, when a user drags and drops a fireplace from theset of fireplaces 1042 onto the structural design layout, if the back ofthe fireplace extends outside of the structural design layout, the usermay be prompted to inform the system whether system whether she wantsthe exterior siding of the structure to extend around the fireplace (orchimney) or whether she wants exposed brick on the exterior of thebuilding. In other implementations, the user may be prompted to decidewhat to do with fireplaces that extend outside in an exterior 3Dstructural design layout mode. In FIG. 53, the user may choose not toadd stairs and continue to FIG. 54, where set of closets and pantries1062 may be suggested to the user. In some implementations, closets andpantries may be considered structural elements, while in otherimplementations, these may be inserted automatically into product designlayouts as product elements.

At FIG. 54, the user may choose to “Add another Floor Level” 1064. In animplementation, if this option is selected, the structural designsketchpad may show the user a tracing layer where the user may bepermitted to fit a second level room or set of rooms to all or a portionof the outside perimeter of the initial floor. If the additional floorlevel is a second or higher floor, the user may still be allowed to usedimensions that extend over the perimeter of the lower floor(s), such asin, e.g., a split-level home where second story walls may extend outbeyond the first floor walls. In other implementations, the system mayby default provide a floorplan matching the lower (or higher) existinglevel(s) and permit the user to edit the structural design layout(s) forthat floor.

In FIG. 54, the user may choose not to add another floor level and mayinstead go to the next step, which is an exterior 3D structural designlayout view, as shown on website 1070 of FIG. 55. In an implementation,the design process continues from defining a structure using 2D drawingsand layouts, somewhat from an interior perspective, to defining the lookand feel of the exterior of the structural design layout that was builtand modified in 2D mode. In an implementation, the exterior 3Dstructural design layout view may provide a 3D modeling system thatbuilds or selects search results and suggestions based on a criteriasuch as design rules, including architectural design rules; or adaptivelearning based on neural network analysis of user design choices, orboth. In an implementation, the system may also draw on pre-existinguser designs.

As shown in FIG. 55, the system may build an exterior 3D structuraldesign layout 1072 that is based on the 2D structural design layout 1068(including a kitchen and a dining room) shown in FIG. 54. As shown inFIG. 55, the new exterior 3D structural design layout 1072 includes adoor 1088 and a chimney 1092 in the appropriate locations (based on theprior 2D drawing 1068 from FIG. 55). In an implementation, the systemmay prompt the user to define a roof to apply to the structure, such asa standard “gable end” roof (not shown) or a “hip” roof 1074. In animplementation, the system may automatically apply a roof such as roof1074 to a 3D layout of the two room structure (constructed based on thelayout 1068) to build the exterior 3D structural design layout 1072.Although in FIG. 55, the structure (e.g., a home), only has two roomsand one floor, any number of rooms and floors are possible.

In an implementation, the focus of the exterior 3D structural designlayout view may be not only on presenting exterior structural elements,but also may be on exterior structural elements that may not ordinarilybe capable of being rendered in 2D drawings of layouts.

As described above, the system may present the user with results orsuggestions (based on, for example, design rules and neural networkanalysis of user design choices) such as “Suggested Entries” 1076. Thesystem may suggest various entry styles (such as entry 1078) based on,for example, the roof type previously selected by the user; on designgoals or search terms (e.g., architectural or interior designinformation, desired styles, etc.) entered by the user on this page orprevious screens.

According to the example progression of the design process, the user maychoose the entry 1078 to apply the entry 1078 to apply to the exterior3D structural design layout 1082. The entry may then appear as the entry1086 in FIG. 56, where on webpage 1080, the user is provided with“Suggested Alterations” 1084. A first alteration 1094 to the exteriorstructure includes windows located on either side of the chimney 1092; asecond alteration 1096 includes four windows; and third alteration 1098includes two windows and a doorway with a deck outside of the kitchen.The suggested alterations (such as exterior design styles) may be basedon, for example, design rules, neural network analysis of user designchoices, or the designs of other users, and the like. For example,architectural standards or guidelines implemented as design rules mightrecommend a window on each side of a fireplace.

In FIG. 57, the user may choose finishes for the exterior of thestructure and may be provided with “Suggested Textures” 1102, such redbricks 1104 (for, say, a traditional style home) or gray shingles 1106(for, say, a New England style “beachhouse”).

The system may suggest more complex exterior structural changes to bemade to the exterior 3D structural design layout 1082, depending on, forexample, choices made by the user. For example, the user might have beenasked what style of home she wanted and respond responded that shewanted a “beachhouse”, and thus may have received results or suggestionsthroughout the design process that tend subtly or more pointedly in thedirection of a “beachhouse”, but choices made by the user during thedesign process might suggest a different structure, architectural style,or exterior design altogether, such as a traditional Colonial, and thesystem may be configured to readjust results or suggestions that may bepresented to the user according to subtle shifts in user choices.

According to the example progression of the design process, the userselects “Next Step” to move to the webpage 1120 of FIG. 58, where theuser may begin “Step 2: Choose Product Layouts.” If the user made anychanges in the exterior 3D structural design layout view that wouldcause the dimensions of the 2D structural design to change, then theuser may be prompted, before proceeding to choose product layouts, toconfirm that the changes should be applied to the 2D structural designlayout (or to any previously defined product layout). The user may alsobe presented with an “Aerial View” 1122 of the 2D structural designlayout 1124 that includes the kitchen and the dining room so that theuser may “Select a Room to Finish the Design” 1126. According to theexample progression of the design process, the user may select thekitchen and in FIG. 59 the aerial view disappears, showing a structuraldesign layout 1142 of the kitchen. In an implementation, the system maydirect the user to cycle through each room in the house, in thisinstance beginning with the kitchen, and then, when that design iscomplete (for example, when one or more product layouts are selected, orwhen a 3D design layout is completed), the user may be directed tochoose another room (here, the dining room is the only other roomdefined in the structure).

In FIG. 59, a screenshot of the webpage 1140 shows a product layoutinterface where the system may suggest a set of product layouts 1144 tothe user to apply to the structural design layout 1142 of the kitchen.In contrast to, for example, the product layout search interface shownin FIG. 42, in FIG. 59, the dimensions of the structural room layout,coupled with the likelihood that the layout is merged with one or moreother rooms (here only a dining room, see FIG. 58), may place morelimitations on the type of product layout searches that may beperformed. For example, searches of pre-existing user designs may provelimited since any matching product layout, may need to have matchingstructural elements in the same locations, as well as to have either thesame dimensions (or be scalable to the same dimensions) as thestructural design layout 1142. Therefore, in an implementation, thesystem may dynamically build a set of product layouts that match thedefined structural design layout 1142, based on, for example, designrules (such as interior design guidelines or standards implemented asdesign rules), or neural network analysis of the design choices of otherusers. In an implementation, the product layouts may typically be builtautomatically by the system rather than manually by a user. There may beany number of scenarios under which the system may dynamically buildproduct layouts, such as when the system provides product layout resultsor suggestions that match or may be scaled to match an existingstructural design layout. Another example scenario may occur when a userhas made changes to the exterior structure; then, the system may, insome implementations conditioned, on confirmation from the user,dynamically build a new product layout based on the changes or mayprovide additional matching product layout results or suggestions to theuser.

FIG. 60 is a diagram illustrating an example of how the system maydynamically build a pair of product layouts 1150, 1154 from a respectivepair of structural design layouts 1142, 1146.

As shown in FIGS. 59 and 60, the structural design layout 1142 includesa closet as a structural element. A parallel structural design layout1146 (shown in FIG. 60) might not include a closet 1160 as a structuralelement. In an implementation, closets may be treated as structuralelements and added to structural design layouts prior to the productlayout process. In an implementation, closets may be treated as productelements and may be added by the system during the product layoutprocess, for example, before, along with, of after other productelements are inserted into the structural design layout to build aproduct layout. According to some design philosophies that may beimplemented by the system as design rules, closets or other storageareas may be considered less structural than, for example, a window or adoor because storage size may be more dependent on everything else thatgoes into a room. According to other design philosophies that may beimplemented by the system as design rules, closets may be consideredmore structural, such in the case of a bedroom, or a pre-existingstructure that already has built-in closets.

According to a first example procedure 1156, product elements such as asink 1162, a stove 1164, and a refrigerator 1166 may be inserted intothe structural design layout 1142 (which includes the closet 1160) inconformance with any structural elements already in the structuraldesign layout 1142 to form an intermediate product layout 1148.Similarly, product elements such as the sink 1162, the stove 1164, andthe refrigerator 1166 may be inserted into the structural design layout1146 along with a closet 1168 that may be identical to the closet 1160.

In an implementation, architectural or interior design guidelines thatmay be implemented as design rules by the system may direct or guide theinsertion of the product elements 1162, 1164, 1166 into the structuraldesign layouts 1142, 1146. Some examples of design industry guidelinesor standards that may be applied to a kitchen may include the following:

(1) sinks typically go below windows; and

(2) core product elements such as a sink, a refrigerator and a stove maybe most advantageously positioned in a kitchen to form a “work triangle”of the products, sized so that, for example, the products may be locatednear to one another and a person using the products may walk to allthree easily.

In an implementation, once product elements such as core productelements have been inserted into a structural design layout, the systemmay insert additional product elements that are more flexibly sizedaccording to product elements and structural elements already in theroom. In the case of a kitchen, such additional product elements mayinclude kitchen cabinets, or an island located in the middle of akitchen, for example. The system may automatically populate suchadditional product elements around and in conformance with structuralelements and other product elements already in the room based oncriteria such as design rules, neural network analysis of the designchoices or behavior of other users of the system, or pre-existing userdesigns.

For automatic insertion and population of kitchen cabinets (such ascabinets 1170, 1172) into a structural design layout (such as layout1148 or layout 1152) around and in conformance with structural elements(such as windows and possibly a closet) and other product elementsalready in the structural design layout (such as a sink, a stove, or arefrigerator and possibly a closet), the following non-exhaustiveexamples of design industry guidelines or standards may be applied:

-   -   kitchen cabinets generally have two levels, and may be referred        to a upper level kitchen cabinets and lower level kitchen        cabinets;    -   upper level kitchen cabinets may not be applied across windows;    -   upper level kitchen cabinets may be spaced at standard distances        from windows so as to not crowd out the windows;    -   The width of a particular upper level kitchen cabinet may match        the width of the lower level kitchen cabinet directly below the        upper level kitchen cabinet, likewise with kitchen cabinet        doors;    -   Silverware drawers may be placed near a dishwasher, if located        in the room;    -   Cabinets for pots and pans may be placed below a cook top range;    -   Lower level kitchen cabinets located immediately below sink may        have two doors, which may allow for a wider cabinet below the        sink; and    -   Industry standard widths for cabinets may apply, depending on        location of the cabinets in the room.

In an implementation, the system may also automatically lay countertopsacross the lower level kitchen cabinets in the 2D product layout, basedon knowledge of where the kitchen cabinets are located.

Of course, the example procedures shown in FIG. 60 may be applied to anyvariety of kitchen layouts as well as to layouts for other rooms thankitchens. The following are merely examples that are not intended in anyway to exhaust the possibilities for dynamically building productlayouts:

For example, for a bathroom, core product elements may include atub/shower, a sink, and a toilet and these may be inserted aroundstructural elements in a 2D structural design layout based on designguidelines that may be implemented as design rules, such as that awindow may not be located in a shower stall.

For example, for an entertainment room, core product elements mayinclude a television. The television product element may be inserted inan ideal location based on the structural design layout. If the roomincludes a fireplace (a structural element), that may serve as anotherfocal point for the room. In an implementation, once certain productelements have been inserted into the structural design layout inconformance with any structural elements in the room, product elementssuch as furniture may be automatically inserted into the structuraldesign layout of the entertainment room. A set of furniture may beapplied, for example, around the television for convenient viewing ofthe television, or, around the television and in a quieter area of theroom. A structural element such as a fireplace (typically alreadydefined in the structural design layout) may be a competing focal pointof the room, so that sets of furniture may likewise be automaticallyplaced for convenient viewing of the fireplace. The system may determinethe approximate size of furniture that may go in these locations,depending on, for example, the overall size of the room. As with otherrooms, interior design styles such as a preference for contemporary ormodern architecture vs. traditional architecture as shown by, forexample, the type of home, may influence the choice of furnitureincluded in the product layout. A “sectional” or “l-shaped” couch may beused for contemporary or modern style homes while a more traditionalcouch may be used in a product layout for a traditional home such as aColonial style home.

For example, for a dining room, a product element such as a dining roomtable may be inserted into a structural design layout of a dining roomand may be automatically located and sized according to the roomdimensions. For example, a long dining room table may be rotated to goalong with the longer length of the room, unless, for example, the roomhas equal sides, or the dining room is an extremely large room. Ingeneral, the system may automatically find an appropriate size for thetable in the room. If the room is an extremely large room, then thesystem may view the room in terms of a dining area and a sitting area,and focus its attention on the dining area when positioning the diningroom table.

For example, for a bedroom, the system may insert a product element suchas a bed into a structural design layout of a bedroom in conformancewith structural elements, including doors or windows. Positioning thebed across from doors or door openings so that entry into the roomtypically looks toward the bed may be considered in compliance with aninterior design guideline that may be implemented as a design rule.Similarly, product elements such as nightstands may be placed underneathwindows (structural elements) on each side of a bed.

As described above, in an implementation, design rules may includeinterior or architectural design guidelines, recommendations, standards,norms, methodologies, philosophies, or styles, individually or incombination, implemented as design rules in, for example, algorithms ofthe system. The design rules may be updated from time to time to reflectchanges or advancements in, for example, interior and architecturalstyles. While algorithms may be based on design rules, in someimplementations the design rules may be guidelines or considerationsrather than strict rules.

Referring once again to FIG. 59, the product layouts 1156, 1158dynamically built in FIG. 60 may be presented as a part of results orsuggestions 1144 to the user. According to the example progression ofthe design process, the user may select the product layout 1156 so thatproduct layout 1156 may be automatically applied to the structuraldesign layout 1142 as shown on the screenshot of the webpage 1200 ofFIG. 61. The product layout 1156 may become a first product layoutoption 1202 on the left-hand side of the screen. The user may choose analternative product layout 1204, which, on the webpage 1220 of FIG. 62may become a second product layout option 1206 for the user. In general,the system may allow the user to retain multiple product layouts for usein the design process.

In an implementation, once a user has chosen one or more product layoutsfor the particular room, an aerial view may be launched (not shown,similar to the aerial view 1122 in FIG. 58) to prompt the user to selecta new room to choose product layouts for. According to the exampleprogression of the design process, the user may decide to skip thedining room and proceed to choosing one or more product sets for the oneor more product layouts, as shown in the screenshots of the webpages1240, 1300 of FIGS. 63 and 64. The above discussions with respect toFIGS. 42 and 43 apply to FIGS. 63 and 64, including discussions ofchoosing product set styles and product sets, and relevant aspects ofthe discussion of the product set search interface in FIG. 38 may alsoapply to FIGS. 63 and 64. Once one or more product sets are chosen bythe user to apply to one or more product layouts, the user may proceedto what may be a finalization of the design process shown in ascreenshot of a webpage 1350 from FIG. 65. The above discussions withrespect to FIGS. 45-47 apply to FIG. 65, including discussions of the 3Ddesign layout and presenting the user with complementary productsuggestions.

FIG. 66 shows an example of a multistage design process. At one stage1402 of the process, in a 2D mode, a user may define a structure in 2Das structural design layouts for rooms, floor plans, or multiple levels,for example, using 2D layouts or drawings, and may modify the structuraldesign layouts by applying structural design elements. At another stage1404, in a 3D mode, a user may refine or modify the structure defined bythe 2D structural design layouts in 3D by applying exterior structuralelements (such as different appendages and textures). A user may chooseproduct sets to be applied to 3D design layouts at this stage as well.At another stage 1406, in a 2D mode, a user may define product layoutsof rooms using 2D layouts or drawings. At another stage 1408, in a 3Dmode, a user may finalize product decisions and may define “productsets” using 3D design layouts.

As shown in FIG. 66, although the multistage design process may proceedfrom left to right in a linear fashion, a user may move from one stageto another stage as desired. For example, a user may define a productlayout of a room at one stage 1406, and then may cycle back if desiredto a prior stage 1404 to modify an exterior 3D structural design layout.Changes made by the user at stage 1404 to the exterior 3D structuraldesign layout, such as moving a window, may cause changes to an alreadydefined product layout and, in an implementation the system may promptthe user at stage 1406 to redefine a product based on the changes. Ingeneral, a user may move back and forth as desired between the stages1402, 1404, 1406, 1408. In an implementation, changes made in thevarious modes and stages may be automatically implemented and applied toall designs in all modes, while in other implementations user may beprompted to respond to the effect that changes in one mode or stage maycause to designs in other modes or stages.

FIGS. 67-73 show screenshots of webpages following a user's exampleprogression through a multistage design process in accordance with thestages 1402, 1404, 1406, 1408 of FIG. 66. The design process takes theuser in broad terms from being presented with structural elements withwhich to design and modify the structural design layouts of one or morerooms (FIG. 67; stage 1402); to being presented with exterior structuralelements (e.g. exterior styles) to modify an exterior 3D structuraldesign layout (FIGS. 68-69; stage 1404); to being presented with productlayout results or suggestions and choosing a product layout (FIGS.70-72; stage 1406); to being presented with product set results orsuggestions and choosing a product set to apply to the product layout(not shown, stage 1406); and to viewing a 3D design layout with theproduct sets included and being presented with complementary productresults or suggestions to choose from to add to the 3D design layout(FIG. 73). The process may be configured so that a user may cycle at anypoint to an earlier or later stage in the design process.

In implementations of the design process shown in FIGS. 67-73, atvarious stages the system search engine may employ redundancy avoidanceor minimization techniques, as described herein, to present more uniqueand potentially more helpful results to the user. The search results orsuggestions may include layouts, elements, or products, from particularusers of the system; however, in other implementations, the designsearch interface may dynamically build design results or suggestions(based on, for example, design rules) for presentation to a user, ratherthan drawing on pre-existing designs. A variety of criteria may beconsidered in building, selecting or presenting results or suggestionsin a particular order.

In an implementation, the design process shown in FIGS. 67-73 mayprovide a user with single screen interface with which to move back andforth between various stages of a design (e.g., stages 1402, 1404, 1406,1408 of FIG. 66) and back and forth between 2D and 3D design and viewmodes. An example of the basic interface is shown in the screenshot ofthe webpage 1430 shown in FIG. 33. Two large tabs 1432, 1434 appear onthe screen. A user may begin to define the interior and exteriorstructure of a design by clicking the first tab 1432 labeled “Define theStructure & Exterior”, while the user may begin to define the interiorlook and feel of a design by clicking second tab 1434 labeled “DefineRoom Layouts & Make Product Selections.”

On the part of the interface shown in FIG. 67, the system is in 2Ddesign mode (and stage 1402 of FIG. 66), the first tab 1432 is activeand a user may drag and drop a rectangle shaped room 1436 from a set ofstructural elements 1438 (here different shaped rooms). The user may beprompted to name the room (here a “Bedroom” and may be, for example,prompted to enter room dimensions. The user may also click in and typein dimensional values. The system may automatically resize the 2Ddrawing or structural design layout 1440 according to the dimensions.The user may draw from additional sets of structural elements (notshown) such as windows, door or door openings, fireplaces, sets ofstairs, and closets or pantries, for example. The sets of structuralelements may be presented as results or suggestions to the user based onany of a variety of criteria, such as design rules, the designs of otherusers, adaptive learning based on neural network analysis of the designdecisions of other users and the like. As more structural elements areadded, the results or suggestions may be dynamically refined to reflectthe modified structural design layout. A user may add other rooms andfloor levels to the design as desired.

On the part of the interface shown in FIG. 68, the system is in 3Ddesign mode (and stage 1404 of FIG. 66), and the first tab 1432 isactive. In FIG. 68, the system may build an exterior 3D structuraldesign layout 1452 that is based on the 2D structural design layout 1440shown in FIG. 67. As shown in FIG. 68, the new exterior 3D structuraldesign layout 1452 includes, for example, a door 1454 and windows 1456in the appropriate locations (based on the prior 2D drawing 1440 fromFIG. 67). In an implementation, the system may prompt the user to definea roof to apply to the structure, or may automatically apply a roof to a3D layout of the multi-level, multi-room structure (constructed based onthe layout 1440) to build the exterior 3D structural design layout 1452.

The structural design layout 1452 exhibits an exterior style 1456 thatmay include various exterior structural elements, for example, exteriorstructural elements that may not typically be capable of being renderedin 2D drawings of layouts. The system, as in FIG. 68, may present theuser with results or suggestions (based on, for example, design rulesand neural network analysis of user design choices) such as “SuggestedExterior Styles” 1458. The system may suggest various exterior stylessuch as exterior style 1460 (including a deck) based on, for example,the roof type previously selected by the user; on design goals or searchterms (e.g. architectural or interior design information, desiredstyles, etc.) entered by the user on this page or previous screens ofthe interface.

The user may, for example, choose the suggested exterior style 1460 thatincludes the deck to apply to the exterior 3D structural design layout1452. The suggested exterior style may be applied to the layout 1452 andthen appear in FIG. 69, on webpage 1470, as the active exterior style1472 on a modified layout 1474. The previously applied exterior style1474 may be shown as a saved option for the user to return to throughoutthe design process.

On the part of the interface shown in FIG. 70, the system is in 2Ddesign mode (and stage 1406 of FIG. 66), the second tab 1432 is activeand a user may begin to define product layouts for different rooms ofthe structure. If, for example, the user made any changes in the 3Ddesign mode to the exterior 3D structural design layout that would causethe dimension of the 2D structural design to change, then the user maybe prompted, before proceeding to choose product layouts, to confirmthat the changes should be applied to the 2D structural design layout,or to any product layouts that the user may have already defined for therooms in an earlier visit to this part of the interface. For example, ifa product layout had already been chosen for a kitchen with a windowfacing out of the structure and kitchen cabinets on either side of thewindow, if the user deleted the window in the 3D design mode (and stage1404 of FIG. 66, e.g. FIG. 68) then the system may, for example,automatically add more cabinets in place of the window and may ask theuser to confirm the change. If the user had added a window to the wallof the kitchen in the exterior 3D design mode, then the system may, forexample, automatically remove kitchen cabinets to make room for thewindow and may ask the user to confirm the change.

The user may also be presented with an “Aerial View” 1502 of all or aportion 1504 of the 2D structural design layout so that the user maychoose a room for which the user may define product layouts. Accordingto the example progression of the design process, the user may selectthe kitchen and in FIG. 71 the aerial view disappears, showing astructural design layout 1522 of the kitchen. The above discussions withrespect to FIGS. 58-62 apply to FIGS. 70-72, including discussions ofdefining and dynamically building product layouts, and choosing productlayouts to apply to a structural design layout of, e.g., a kitchen.After selecting one or more product layouts, the user may proceed (notshown) to choosing one or more product sets for the one or more productlayouts. The above discussions with respect to FIGS. 42 and 43 areinstructive here, including discussion of choosing product set stylesand product sets, and relevant aspects of the discussion of the productset search interface in FIG. 38.

Once one or more product sets are chosen by the user to apply to one ormore product layouts, the user may proceed to what may be a finalizationof the design process shown in a screenshot of a webpage 1580 from FIG.73. On the part of the interface shown in FIG. 73, the system is in 3Ddesign mode (and stage 1408 of FIG. 66), the second tab 1432 is activeand a user may finish filling in products on the 3D design layout, orcycle back to an earlier stage of the design. The above discussions withrespect to FIGS. 45-47 apply to FIG. 73, including discussions of the 3Ddesign layout and presenting the user with complementary productsuggestions.

In an implementation, a graphical image of a home design product suchas, e.g., a piece of furniture, an appliance, or an accessory, may bereceived, and 2D and 3D representations of the home design product maybe created. The 2D and 3D representations may be created or builtaccording to, or based on, any of a variety of criteria ormethodologies, such as design rules; adaptive learning using neuralnetworks that analyze prior user design behavior and selections; data,input, or feedback provided by users or partners of the system; datascraped from websites of, e.g., home design product manufacturers; datafrom databases of saved standard profiles, pre-built 3D base models, andknown attributes of product types and images; results of evaluating thegraphical image, including results from perspective determination, edgedetection, plane detection, feature and profile detection andevaluation; standard solid modeling techniques; and design recipes.

2D and 3D representations of a home design product may be created orbuilt using any of a variety of acts, depending on the implementation,and the acts may be performed in a different order and may be repeated.Processing used to create or build the 2D and 3D representations of thehome design product based on a graphical image of the home designproduct may proceed in parallel. The processing used to create or buildthe 2D and 3D representations may be entirely or partially automated.All or part of the processing may also be manually performedresponsively to input from a user.

In an implementation, design rules may include interior or architecturaldesign guidelines, recommendations, standards, norms, methodologies,philosophies, or styles, individually or in combination, implemented asdesign rules in, for example, algorithms of the system. The design rulesmay be updated from time to time to reflect changes or advancements in,for example, interior and architectural styles. While algorithms may bebased on design rules, in some implementations the design rules may beguidelines or considerations rather than strict rules.

A graphical image or graphical image file, such as a JPEG or a GIF file,that depicts, e.g., a home (or landscape) design product may be receivedby the system. The home design product may be any of a variety ofproducts, for example, a piece of furniture, such as a chair or a couch;an appliance, such as a range, dishwasher, or refrigerator; or any otherfurnishings, appliances or accessories. The system may create or build2D and 3D representations of the home design product depicted in thegraphical image. The system may accommodate a vast variety of graphicalimages and may use one or more techniques to create or build the 2D and3D representations of the home design product.

The graphical image may be a basic image received by the system, forexample, without any content or information associated with the image.The graphical image may be a bookmarked or scraped image that isaccompanied by content or information (such as taxonomy or folksonomyinformation, sizing information, keywords) associated with or regardingthe home design product depicted in the graphical image. The graphicalimage may be received into the system as part of a user bookmarking ahome design product from, e.g., a website of a manufacturer or aretailer of the home design product, as described in more detail above.An advertiser or a manufacturer may provide the graphical image of ahome design product as part of placing content and information regardingthe home design product into the system, via, e.g., a bookmarking tool,as described in more detail above. The system may use a scrapingtechnology and a taxonomy/folksonomy based indexing and categorizingsystem to scrape websites of, e.g., home design product manufacturersand retailers for content and information regarding a home designproduct, as described in more detail above. The graphical image of thehome design product may be part of content received from automaticallyscraping a website.

Several examples of graphical images depicting home design products areshown in FIGS. 74-77. FIG. 74 is a graphical image or picture 1600 of arange 1602, FIG. 75 is a picture 1800 of an armchair 1802, FIG. 76 is apicture 2000 of a kitchen stool 2002, and FIG. 77 is a picture 2100 of aset of shelves 2102 or a shelving unit 2102.

Once the graphical image has been received by the system, the system mayattempt to identify and define the product type, or the product modeltype, of the home design product depicted in the graphical image. In sodoing, the system may retrieve one or more pre-built 3D base models thatcorrespond to the defined or identified product model type of the homedesign product depicted in the graphical image.

The product model type may be identified and defined in a variety ofways. If the graphical image received by or input to the system isassociated with scraped data, from, e.g., the website of a manufactureror retailer of the home design product. In scraping the website, thesystem may have obtained the type of the product, for example anarmchair, and other words related to the specific features of theproduct, for example. The system may know, based on the scraped data,the product category and subcategory and other information, that thehome design product depicted in the graphical image is a bathtub, ormore particularly a five foot long bathtub that comes in standarddimensions. The system may search a database using words or otherinformation related to the product to automatically retrieve one or morepre-built 3D base models.

The system may also prompt a user for input, by, for example, presentingone or more pre-built 3D base models to the user for the user to choosefrom to associate with the graphical image.

The system may also use image evaluation. The system may evaluate theimage by, e.g., using edge detection techniques. As part of imageevaluation, the system may examine past data that the system gatheredrelated to image interpretation, i.e., neural networking data. Forexample, if the system has repeatedly found the same isometric view ofan armchair the system may determine based on this neural networkingdata that a graphical image presently being interpreted (using edgedetection and feature detection) is most likely: say, an armchair andthe system may retrieve one or more pre-built 3D base models of anarmchair. The system may identify edges and surfaces in the graphicalimage, such as the arm of an armchair, to identify the product modeltype as an armchair and retrieve one or more pre-built 3D base models ofan armchair. The system may match edges and surfaces in the graphicalimage with images previously collected by the system, for example. Theproduct model type may be determined based on a feature interpreted fromthe graphical image that is common to a particular product model type.

The one or more pre-built 3D base models retrieved by the system may bemanipulated (e.g., parametrically adjusted, or scaled) to serve as abasic or underlying structure for the 3D representation of the homedesign product. A 3D base model may provide baseline data for theeventual 3D representation of the home design product. For example, thepre-built 3D base model may provide the overall shape of an armchair.The arms may not be rounded, for example, as in an example graphicalimage of an armchair, but the basic shapes and general planes of thearmchair may be similar or the same. A pre-built 3D base model may beused to build a 3D base model from scratch or on the fly that may inturn be used as the basic or underlying structure for the 3Drepresentation of the home design product.

For some graphical images of home design products, there may be nopre-built 3D base model that corresponds to or is associated with theimage and that may, for example, serve as the basic or underlyingstructure of a 3D representation of the home design product. The systemmay, without the use of a pre-built 3D base model, build a 3D base modelfrom scratch or on the fly in an automated process that may in turn beused as the basic or underlying structure for a 3D representation of thehome design product.

Referring to the several examples of graphical images depicting homedesign products shown in FIGS. 74-77, the system may more easily definethe product model type of the range 1602 depicted in picture 1600 ofFIG. 74 than the set of shelves 2102 depicted in picture 2100 of FIG.77, or even the armchair 1802 or the kitchen stool 2002 of the pictures1800, 2000 of FIGS. 75 and 76. Similarly, the system may find one ormore pre-built 3D base models to associate with the range 1602, or eventhe armchair 1802. An example pre-built 3D base model 2200 for the range1602 is shown in FIG. 78. The system may, for example, e.g., find nopre-built 3D base model to associate with the set of shelves 2102, or,for that matter, the armchair 1802. The system may then build 3D basemodels from scratch or on the fly in an a automated process to associatewith the set of shelves 2102 or the armchair 1802. The complexity of theshape of the image of the product may impact whether the product modeltype may be defined, whether a pre-built 3D base model may be found toassociate with the home design product, or whether a 3D base model maybe built from scratch or on the fly in an automated process. Forexample, furniture may present more complexity than a more box shapedappliance, such as a range.

The 3D base model may be built according to a tile recipe. The filerecipe may be adjusted based on any of a variety of criteria ormethodologies, including data and input provided by a user; and adaptivelearning using neural networks that analyze prior user design behaviorand selections, for example.

Assuming that no pre-built 3D base model is found to associate with thegraphical image, for example, the picture 1800 showing the armchair 1802in FIG. 75, or that any pre-built 3D base models that are found are usedin building a 3D base model to associate with the graphical image, thesystem may first determine the perspective of the graphical image of thehome design product. That is, the system may determine the perspectivethat the picture of the home design product, here the picture 1800showing the armchair 1802, was taken in. The system may examine thingsin the background of the graphical image other than the home designproduct to determine the perspective and to obtain perspectiveinformation. The system may obtain image background information likewall and floor intersections, wall and wall intersections, and ceilingand wall intersections. The picture 1800 also shows the intersection ofthe armchair 1802 with walls 1806, 1808 and the floor 1804. The systemmay also examine object data in the home design product itself. Thesystem may draw lines along the home design product image to determineperspective, such as a line from the right front leg to the right backleg of the armchair 1802. Another line may be along the top of thearmchair 1802, and still another may be from the left front leg to theright front leg of the armchair 1802. The different lines in the drawingmay create planes for a 3D base model of the armchair 1802. The systemmay obtain the perspective information to determine how to match a 3Dbase model to the image of the armchair 1802, and to better interpretdata in the image of the armchair 1802.

The system may assign main overall dimensions to the image of thearmchair 1802, such as dimensions for height, width, and depth of thehome design product, if the dimensions are available. The system may usethe main overall dimensions to set a relative scale for a 3D base modelof the armchair 1802. The main overall dimensions may available fromuser or advertiser input or from, e.g., data scraped from a website of amanufacturer or retailer. For example, a retailer typically may provideraw dimensions of a home design product so that people can know whetherthey can, e.g., fit the product into their home, or in a room or spaceof their home, and the system may obtain these dimensions as scrapeddata. If no dimensional information is available, or to complement suchinformation, the system may infer and assign sizing or dimensionalinformation based on information provided in the graphical image, suchas the known size of other objects in the scene of the graphical image,or known dimensions of walls in the scene, for example.

The system may also assign detailed object information, includingdimensional data, for features of the home design product, if thedetailed object information is available. For example, in the case of adresser, the system may have information regarding the size of thedrawers, or the number of drawers, in the dresser. For a sink, thesystem may have information such as the distance between the two holesfor the faucets, or the dimensions of the basin of the sink. On a range,detailed object information may include where the control panel islocated, how many knobs on the control panel, and the like. On anarmchair such as the armchair 1802, the system may have information onthe distance that the front faces of the arms are set back from thefront of the armchair, either absolutely or relative to the distancefrom the front to the back of the armchair.

The system may use edge detection techniques on the graphical image tofind and assign main object planes in the CAD model that is being usedto build the 3D base model. The main object planes may be top, bottom,left, right, front, and back planes that may form a box. Opposing mainobject planes may be parallel to one another. The main object planes maytogether form a bounding box for the CAD model that may define the outerboundaries of the home design product as depicted in the graphicalimage. As such, the main object planes may be used as the basis forbuilding a 3D base model of the home design product. The system mayutilize perspective information to find and assign the main objectplanes.

According to the example of the armchair 1802 in the picture 1800 ofFIG. 75, the outside of the arms may determine the left and rightplanes; the front of the chair may determine the front plane; the top ofa top cushion may determine the top planes; and the legs, in combinationwith the distance between the left and right planes, and front and backplanes may determine the bottom plane, for example. A bounding box 2300for the armchair 1802 is shown in FIG. 79, with top 2306, bottom 2308,left 2312, right 2304, front 2310, and back 2302 planes.

The system may also assign main object planes 2400 that intersect at thecenter of the home design product image and that are perpendicular toone another, as shown in FIG. 80, in the CAD model that is being used tobuild the 3D base model.

The system may resize the bounding box formed by the main object planesaccording to the main overall dimensions described above. The mainoverall dimensions may determine the distance between opposing mainobject planes. The bounding box may define the outer boundaries of the3D base model within which the 3D base model may be built.

The system may next determine intermediary planes, or feature levelplanes, based on, e.g., analysis of the home design product image and onaccumulated knowledge of the system and may assign these feature levelplanes within the bounding box in the CAD model. The accumulatedknowledge of the system may include, for example, data obtained frominterpretation of the image, such as edge detection data, perspectiveinformation, feature detection data and feature information; detailedobject data such as dimensional data for features of the home designproduct; stored feature level plane information for the product modeltype; and neural networking information.

As an example of accumulated knowledge of the system, the system mayknow, based on, e.g., interpretation of the image, the product modeltype, or detailed object data such as dimensional information, on anarmchair such as the armchair 1802, the distance that the front faces ofthe arms are set back from the front of the armchair, either absolutelyor relative to the distance from the front to the back of the armchair.For example, the system may know that the front faces of the arms of thearmchair 1802 are, within the bounding box of the CAD model, ⅕ of thetotal distance from the front of the bounding box to the back of thebounding box, so that a feature level plane may be assigned at ⅕ of thedistance back from the front of the bounding box. The system may haveinterpreted the image using, e.g., edge detection, to determine thatfeatures such as the front faces of the arms were at this distance orrelative distance. The system may have had prior knowledge that inarmchairs, or in this type of armchair, a feature level plane should beplaced at this distance or relative distance. FIG. 81 shows the boundingbox 2300 with a feature level plane 2314 included within the box 2300,at the relative location that the front faces of the arms are locatedwithin the image of the armchair.

The system may place numerous intermediary planes, or feature levelplanes, within the bounding box in the CAD model that is being used tobuild the 3D base model. Each feature level plane may correspond to oneor more features that, e.g., the system detected while interpreting theimage, or that, e.g., the system knew to look for based on knowledge ofthe product model type. The system may also use a pre-built 3D basemodel to know which feature level planes to use. Feature level planesmay correspond to stored known feature level planes for the productmodel type. For example, the system may know that a feature level planeexists for the front faces of the arms of an armchair (see the featurelevel plane 1812 in FIG. 82), that another feature level plane existsfor the top of the seat cushion of the armchair (see the feature levelplane 1814 in FIG. 83), and that another feature level plane exists forthe right side of the base of the armchair that extends up through theright arm of the armchair (see the feature level plane 1816 in FIG. 84).The feature level plane 1812 may be assigned to capture the front facesof the arms of the armchair, the feature level plane 1814 may beassigned to capture the top of the seat cushion of the armchair, and thefeature level plane 1816 may be assigned to capture the right side ofthe base of the armchair Another feature level plane may be assigned tocapture the left side of the base of the armchair. The left side of thebase is not shown in the armchair image, but a plane may be defined inpart based on the left edge that can be seen of the front of thearmchair. The plane may then be rotated until the plane is parallel tothe left main object plane 2312 of the bounding box 2300. The featurelevel planes shown in FIGS. 82-84 may be parallel to different mainobject planes of the bounding box, but for other home design productimages, the feature level planes may be, but need not be, parallel tomain object planes.

The system may recognize features in the home design product image andassign a feature level plane to capture those features. The system mayassign a feature level plane to an image in a location where featureswould be expected to be found based on accumulated knowledge of thesystem.

The graphical image of the home design product may in general be araster graphic, made up of pixels. The system may examine the rastergraphic, or raster graphic information from the graphical image forfeatures in the hone design product image, such as the front faces ofthe arms of the armchair. The system may build a 2D vector drawing, aprofile that may correspond to a feature in the home design productimage. The system may build the profile in a feature level plane. Thesystem may build or create the profile from scratch by detecting thefeature in the home design product image and approximating the featurein a 2D vector drawing. The system may retrieve a profile from adatabase of standard profiles that correspond to features in home designproduct images. The stored profiles may be parametrically adjustable bythe system so that a stored profile may be automatically resized, basedon, e.g. sizing information for the home design product image, to matcha detected feature on the home design product image.

The system may thus apply existing profiles for features of the homeproduct image from the database of standard profiles to the featurelevel planes. The system may parametrically adjust the existing profilesto align with features detected in the home design product image. Thesystem may build profiles for features of the home product image to thefeature level planes, using 2D vector drawing techniques, such asdrawing lines, arcs, curves, and the like.

The system may use edge detection to find edges of features in thefeature level planes that the system has assigned. Candidates forfeatures are parts of the home design product image that intersect thefeature level planes, or that lie within the feature level planes.

In performing feature detection, the system may choose to ignore certainedges detected in the home design product image as extraneous detailsand not include representations of the details in the 3D base model.

The system may create numerous profiles on numerous feature levelplanes. The profiles that the system may create may be identified ascandidates for application of common solid modeling techniques to theprofiles. Common solid modeling techniques may include extruding anobject or a shape (here a profile), extruding a shape along a path (suchas a curved path), revolving a shape, revolving a shape along a path,and other solid modeling techniques well known in the art, such aswarping, chamfers, fillets, drafting, angles, and cuts.

The system may apply common or standard solid modeling techniques to aprofile, i.e., 2D vector drawing, to complete a feature of the homedesign product image. For example, the seat cushion of the armchair maybe modeled in a 3D CAD model by identifying a profile 1818 in thefeature level plane 1814 of FIG. 83 and FIG. 85 that corresponds to theshape (a “T-shape”) of the top of the seat cushion. The T-shaped profile1818 may be extruded downward to form a 3D model 1820 of the seatcushion of the armchair as indicated by the arrows in FIG. 86 (thebounding box and the feature level plane are not shown in FIG. 86).Referring to FIG. 87, a 3D model of the seat cushion may also be builtby identifying a rectangular profile 1822 in a feature level plane 1824that shares the same plane as the front plane 2310 of the bounding box2300 (of, e.g., FIG. 79). The rectangular profile 1822 may be extrudedtoward the back of the armchair (as indicated by the arrows in FIG. 88)to form a box 1826 that serves as a bounding box 1826 for a 3D model ofthe seat cushion. As shown by the dashed lines in FIG. 89, the systemmay then identify the portions 1828, 1830 of the box that overlap partsof the right and left arms of the armchair 1802 and may remove thoseportions 1828, 1830 to form a 3D model 1832 of the seat cushion of thearmchair (as shown in FIG. 90).

Use of either profile 1818 (FIG. 85) or 1822 (FIG. 87) by the system mayresult in an acceptable 3D model of the seat cushion, but the profile1818 may be seen as a “best” profile since the system may have performedfewer acts in less time to build the 3D model 1832 of the seat cushionthan to build the 3D model 1820 of the seat cushion.

As described, the profiles that the system may create may be identifiedas candidates for application of common solid modeling techniques to theprofiles. The system may evaluate the “best” profile for application ofsolid modeling techniques, such as extrusion or revolving. The “best”profile or “best” profiles may be selected according to any of a varietyof criteria, including which profile results in the least execution timeby the system; which profile results in the most complete feature, whichprofile results in the most efficient use of solid modeling techniques;and the like. The system may store “best” profile information as part ofthe database of standard profiles described above. The system may use“best” profiles, or “best” profile information, from the database toselect which of the candidate profiles are the “best” profiles forapplication of common solid modeling techniques. For example, “best”profile information may cause the system to select the profile 1822 forbuilding a 3D model of the seat cushion of the armchair. The system maylack “best” profile information on one or more profiles and so may applysolid modeling techniques absent “best” profile information. “Best” mayinclude the best way, or may include a preferred or better way accordingto any of a variety of criteria.

The system may also determine portions of the images on the featurelevel planes that may be cropped for subsequent application to the 3Dbase model to form the 3D representation of the home design product.

If the home design product depicted in the graphical image is thought tobe symmetrical, or to have symmetrical features, the system may mirrorprofiles or profile information on the feature level planes to parts ofthe plane where a symmetrical features intersects, or is expected tointersect, the feature level plane. F, or example, the system mayrecognize the armchair 1802 as having the same arms, one on the left,and one on the right. A profile 1834 that may define the front face ofthe right arm of the armchair (including the portion of the front facecovered by the seat cushion) in the feature level plane 1812 (see FIG.91A) may be mirrored around an axis 1836 in the center of the plane 1812to form a profile 1838 that may define the front face of the left arm ofthe armchair in the plane 182. The system may use a mirroring techniqueto provide profiles for symmetrical features that may not be shown, orthat may be partially obstructed due to the perspective in which thehome design product image is set. The system may mirror profiles priorto application of solid modeling techniques to the profiles, such asextrusion of a profile. The system may mirror, e.g., an extruded profileto form a copy of an extruded profile, with the mirroring occurringequidistant from one side of a plane to another side of the plane,rather than within a plane.

The system may apply the common solid modeling techniques, such asextrusion and revolving, to profiles that have been evaluated andidentified. The system may apply the techniques to “best” profiles andmay use techniques that have been evaluated for those profiles. Forexample, the system may extrude a profile such as the profile 1818 ofFIG. 85 to build the 3D model 1820 of the seat cushion of FIG. 86. Inthe case of an image of a lamp, for example, a profile of the lamp maybe revolved around as part of building a 3D base model for the lamp. Thesystem may apply mirroring techniques to profiles before or afterapplying common solid modeling techniques such as extrusion andrevolving. The system may prompt a user for input regarding whether thesystem has interpreted the home design image correctly or acceptably tothe user. For example, the system may request a user to specify whetherone or more profiles are appropriate for application of solid modelingtechniques. The system may be fully or partially automated in building a3D base model. The system may perform acts toward building a 3D basemodel to associate with the home design product image without requestingor receiving input from a user.

The system may apply other common solid modeling procedures to objectsupon which some common solid modeling procedures have been performed.For example, a profile may have been extruded to form a 3D model of aparticular feature. The system may apply solid modeling procedures suchas applying chamfers, fillets, drafting angles, and cuts to 3D models offeatures of the home design products, such as, say, the base, a seatcushion, or the arms of the armchair. Feature details detected via,e.g., edge detection of the home design product image or accumulatedknowledge of the system may inform the application of procedures suchapplying chamfers, fillets, drafting angles, and cuts to and into 3Dmodels. For example, the front edge of the seat cushion may be rounded,so that system may apply fillets to the front edges of the 3D model 1820of the seat cushion to round the edges of the seat cushion. A chamfermay include a flat edge cut out of a corner, a drafting angle mayinclude an angle of a surface, and a cut may include a portion cut outof the 3D model. The system may prompt a user to apply the solidmodeling techniques, or the process may be automated, or a combinationof both.

The system may apply warping techniques to warp 3D model surfaces aspart of building the 3D base model. The system may warp extrusions ofprofiles based on feature edge detection and accumulated knowledgeregarding properties of the home design product. For example, the armsof a stuffed armchair may be puffy due to stuffing under the fabric,making the surfaces of the arms along the length of the arms curvedrather than flat. Likewise, the cushions of an armchair may be stuffedand thus the surfaces of the cushions are curved or warped rather thanperfectly flat. The system may warp the extrusions of profiles thatformed the 3D models of the arms and the cushions of the armchair. Thebase of the chair may have flat surfaces and may not need warping, forexample.

The system may also determine an order in which to apply solid modelingtechniques to profiles (or, e.g., extrusions of profiles) and otherwisebuild the 3D base model to associate with the home design product image.The order of building the 3D base model may be based on, for example,standard industry design techniques or design rules; and previousknowledge accumulated regarding efficient or useful methods of buildingthe 3D base model for the product model type. For example, theaccumulated knowledge of the system may indicate that an efficient wayto build the 3D base model of an armchair may be to draw or model thebase of the chair, then to draw the arms of the chair, then to draw theback of the chair, then to draw the cushions (seat and back) of thechair, with the legs of the armchair being drawn at any point after thebase is drawn. The system may also apply known solid modeling techniquessuch as warping surface at various points in the building of the 3D basemodel, for example, the system may warp the surfaces of the arms beforethe cushions. The system may prioritize features, such as base of thechair before the arms of the chair, and arms of the chair beforecushions of the chair, for example. In general, the system may build the3D base model using any order of solid modeling techniques as applied toany order of profiles, features, or 3D model objects.

The system may compare the resultant built 3D base model with any of avariety of things to determine whether the resultant built 3D base modelproperly characterizes the home design product image and may thus beassociated with the home design product image and used as the basic orunderlying structure for the 3D representation of the home designproduct. The system may compare the built 3D base model with the homedesign product image using, e.g., edge detection techniques; to data,scraped from a website of a manufacturer or retailer of the home designproduct; to one or more pre-built 3D base models for, e.g., the productmodel type, such as an armchair; to past data that the system gatheredrelated to image interpretation, i.e., neural networking data relatingto, e.g., the product model type. The system may present one or morebuilt 3D built 3D base models to a user for the user to verify, or tochoose from to associate with the graphical image. A user may beprompted to choose a model that most closely approximates the homedesign product image, and may given the option to ask the system to goback and build a new 3D base model. The system may have built several 3Dbase models serially or in parallel for presentation to a user or forcomparison with the home design product image.

If the system determines, for whatever reason, that the built 3D basemodel does not properly characterize the home design product image, thesystem may return to previous acts of the process for building the 3Dbase model and, for example, repeat actions until the system validates a3D base model, i.e., obtains a built 3D base model that is a closermatch to the home design product image. The system may, for example,return to the edge or feature detection process, or main object orfeature level plane assignment or profile identification, to find apoint at which the process for building the 3D base model may have goneawry. The 3D base model comparison and validation process may be fullyautomated, or may be partially automated interspersed with prompts to auser, for example.

The system may build 3D base models of non-symmetrical home designproduct images as well, or of partially non-symmetrical home designproduct images. The system may employ iterative best guess techniquesthat may leverage any of, e.g., parallel processing, imageinterpretation, accumulated knowledge of the system regarding the homedesign product or detected features of the home design product, and userinput, or a combination of these.

When the system compares and attempts to validate a 3D base model, thesystem may present to a user for verification alternative 3D base modelsthat are based on non-symmetrical interpretations of the home designproduct image.

Once the system has built a 3D base model, the system may examine thesurfaces of the 3D base model and evaluate potential groupings ofsurfaces, or subsets of surfaces, for textures to be applied to. Thesystem may evaluate the potential groupings based on any of a variety offactors, such as, data from pre-built 3D base models; imageinterpretation data including edge detection data from the home designproduct image; commonly used surface groupings for the product modeltype; scraped data; user input or prior user feedback; neural networkingdata; or any combination of these. Depending on the particular 3D basemodel associated with the home design product image, the 3D base modelmay be quite complex, with numerous surfaces for textures to be appliedto. For example, for the armchair 1802 shown in the graphical image 1800of FIG. 74, the built 3D base model may have numerous surfaces, withflat edges, and the like. Several of the surfaces may share the samematerial, and, depending on the home design product in question, thematerial may need to match up at the edges of surfaces. This may be seenmore clearly in the case of a striped armchair, where the built 3D basemodel may have numerous surfaces, but several surfaces may be groupedtogether to form surface groups so that textures applied to the surfaceswill match at the edges, for example. For example, stripes on a backcushion may be designed to match up with stripes on a seat cushion andthe two surfaces may in some situations be treated as a single surfacegroup for purposes of applying textures. A slipcover on a couch may bemodeled as different surfaces in a 3D base model, but the surfacesmaking up the slipcover may in some situations be treated as a singlesurface group for purposes of applying textures. The system may leverageedge detection data used to form surfaces of the 3D base model to definewhere textures may be applied. The surface groupings (or surface subsetgroupings) may vary considerably depending on the particular home designproduct being modeled, or on the portion of the product being modeled.For example, for the (typically) pre-built 3D base model 2200 (see FIG.78) of the range 1602 of the picture 1600 (see FIG. 74), the surfacesmay less complex, being, for example, the front, top, panel, and sidesurfaces. Surface groupings in a 3D base model may depend on whether thesurfaces in question appear in the home design product image, or are notshown or are partially obscured in the image.

The system may evaluate techniques to map textures onto the 3D basemodel. The textures may include, for example, cropped images from thehome design product image or stored textures, such as proceduraltextures, that may not be part of the home design product image. Aprocedural texture may include a form of texture that is typically asquare and is designed so that when the squares are placed together,e.g., the top of one texture may match the bottom of another texture, sothat the squares may map properly. In the case of the armchair 1802shown in the picture 1800, if the armchair is made of a solid whitecloth material, the procedural texture corresponding to the material maybe white cloth. Certain cropped images from the home design productimage may also be used as approximate procedural textures and mappedtogether to apply to a surface or surfaces.

Texture mapping techniques to be evaluated by the system may includetechniques such as tiling, warping and stretching. Tiling a texture mayinclude repeating the same texture pattern, usually a square texture,although other shapes may be used over and over again to fill in asurface. The texture pattern may map properly like a procedural texture,although other textures may be used. Stretching a texture may includestretching a cropped image or texture to meet the confines of aparticular surface or surfaces. Warping a texture may include applying atexture to fill in and smooth out rough internal edges of the 3D basemodel. For example, in a 3D base model of a sink or a tub, the portionof the model that includes the basin of the sink or the inside of thetub may include rough edges at the bottom of the sink or the inside ofthe tub. The system may use tessellation techniques of varyingcomplexity in 3D modeling to approximate rounded looking shapes so thata 3D base model does not include rough internal edges, but the systemmay also, or instead, use texture warping effects to remove hard anglesof the 3D base model.

The system may evaluate the potential texture mapping techniques basedon any of a variety of factors, such as, data from pre-built 3D basemodels; image interpretation data including edge detection data from thehome design product image; commonly used surface groupings for theproduct model type; scraped data; user input or prior user feedback;neural networking data; or any combination of these.

The system may extract texture data from the home design product imagein the form of cropped images. The cropped images may be shapes that maybe tiled, stretched or warped on to the surfaces of the 3D base model.The system may assemble the cropped images from the home design productimage onto profiles on the main object planes and the feature levelplanes. The system may retrieve stored textures, such as proceduraltextures, that may not be part of the home design product image. Thestored textures may be tiled or warped on to the surfaces of the 3D basemodel. Textures may be extracted and retrieved automatically or may be,e.g., selected by a user.

The system may group the surfaces based on the evaluated surfacegroupings. The system may apply the extracted textures and retrievedstored textures to the surface groupings and individual surfaces usingthe evaluated texture mapping techniques, such as warping, tiling andstretching. The system may apply material properties such as reflectanceto the surfaces. For example, a 3D based model of a dresser with amirror attached may have reflectance applied to a surface correspondingto the mirror.

Less complex 3D base models may include a combination of stretchedcropped images on surfaces along with applied procedural textures. Morecomplex 3D base models with a multiplicity of surfaces may have moreprocedural textures applied using tiling and warping to the surfaces inthe 3D base model, although on some surfaces stretched cropped imagesmay be used. The system may use a recipe for applying textures to the 3Dbase model. In general, any combination of textures, including croppedimages from the home design product image and stored textures, may beused with any combination of texture mapping techniques, includingtiling, stretching, and warping, to apply to any combination ofsurfaces, subsets of surfaces, and groups of surfaces in 3D base models.

The system may evaluate the 3D base model with the textures applied andmay determine whether to apply textural details to surfaces of theimage. Textural details may include bump maps or transparent textures,which may include grayscale images typically having light and dark partsof varying transparency so that any underlying texture appears throughthe bump map or transparent texture to varying degrees. Depending on theparticular bump map applied to a surface, the surface may appear to be,e.g., fluffy, furry, shagged, and the like. That is, the bump map ortransparent texture applied to a surface may give the visual illusionthat the surface has detailed features. Other standard textural detailsand textural detail techniques may be applied.

With any textural details and textures applied to the 3D base model, thesystem may use the resulting 3D representation of the home designproduct shown in the graphical image to create a 2D drawing, or 2Drepresentation of the home design product. The system may use a top todown view of the 3D representation to create the 2D drawing. Example 2Ddrawings for the home design product images shown in FIGS. 74-77 areshown in respective FIGS. 92-95. A 2D drawing 2500 of the range 1602(see FIG. 74) is shown in FIG. 92. A 2D drawing 2600 of the armchair1802 (see FIG. 75) is shown in FIG. 93. A 2D drawing 2700 of the kitchenstool 2002 (see FIG. 76) is shown in FIG. 94. A 2D drawing 2800 of theset of shelves 2102 (see FIG. 77) is shown in FIG. 95. In animplementation of the home design tool, a 2D drawing or 2Drepresentation may be used, for example, in a 2D layout of the homedesign tool, and may be converted to the 3D representation when, forexample, a user switches from a 2D mode to a 3D mode in the home designtool. An example 3D representation 2900 of the range 1602 (see FIG. 74)(with textures applied to the 3D base model 2200 of FIG. 78) is shown inFIG. 96.

In an implementation, the system may use a graphical image of a homedesign product itself as a 3D representation of the image, withoutresort to a 3D base model. For example, sizing information (such asheight and width in a 2D plane) for a home design product depicted in agraphical image of a home design product may be obtained from sourcessuch as scraped data from a website of a manufacturer or retailer of thehome design product, user input, perspective information determined fromthe graphical image, edge detection, feature detection, and surfaceinformation, and the like. The kitchen stool 2002 depicted in thegraphical image 2000 shown in FIG. 76 may be used to illustrate anexample of this technique. The height and width, and scaled dimensions,of the kitchen stool may be determined from one or more of the abovesources, such as scraped data. The white areas within the kitchen stoolmay be made transparent using edge detection of the image, resulting ina “billboard” like 3D representation of the kitchen stool that may beplaced in a 3D layout. The system may draw on other dimensionalinformation about the top of the kitchen stool to build a 2D drawing or2D representation of the kitchen stool, such as the 2D drawing 2700 ofFIG. 94. In an implementation of the home design tool, a 2D drawings or2D representation may be used, for example, in a 2D layout of the homedesign tool, and may be converted to the 3D representation when, forexample, a user switches from a 2D mode to a 3D mode in the home designtool. The “billboard” like 3D representation of the kitchen stool may belocated in a 3D layout based on the location of the 2D drawing in a 2Dlayout. The 3D representation may be sized using relative scaledepending on where the 3D representation is located in a 3D layout suchas a room layout. If the 3D representation is located further back in,e.g., a room, the 3D representation may appear smaller due toperspective. The system may apply a technique to stretch the 3Drepresentation dynamically as perspective shifts, for example as a 3Dlayout is rotated. The system may stretch the 3D representationdynamically in such a way that the 3D representation appears to beviewed from the side rather than, from a corner (the kitchen tool 2002is shown from a corner perspective as shown in FIG. 76).

In an implementation, upon receiving a graphical image of a home designproduct such as a kitchen cabinet, a door, a window, and the like, thesystem may build, or otherwise associate with the home design productimage, a box-shaped 3D base model of limited complexity. The system maythen apply textural details and textures (such as a paint color) to the3D base model to form a 3D representation of the home design product. Asdescribed above, textural details may include bump maps or transparenttextures, which may include grayscale images typically having light anddark parts of varying transparency so that any underlying texture (suchas a paint color) appears through it to varying degrees. Depending onthe particular bump map applied to a surface, the surface may appear tobe, e.g., fluffy, furry; shagged, and the like. That is, the bump map ortransparent texture applied to a surface may give the visual illusionthat the surface has detailed features. In the case of a kitchen cabinetor a door, the bump map or transparent texture may give the visualillusion of a wood surface, for example. Other standard textural detailsand textural detail techniques may be applied.

Once a graphical image of a home design product has been received by thesystem, the system may attempt to identify and define the product type,or the product model type of the home design product depicted in thegraphical image. In so doing, the system may retrieve one or morepre-built 3D base models that correspond to the defined or identifiedproduct model type of the home design product depicted in the graphicalimage. Referring once again to the several examples of graphical imagesdepicting home design products shown in FIGS. 74-77, the system may moreeasily define the product model type of the range 1602 depicted inpicture 1600 of FIG. 74 than the set of shelves 2102 depicted in picture2100 of FIG. 77, or even the armchair 1802 or the kitchen stool 2002 ofthe pictures 1800, 2000 of FIGS. 75 and 76. Similarly, the system mayfind one or more pre-built 3D base models to associate with the range1602, or even the armchair 1802. An example pre-built 3D base model 2200for the range 1602 is shown in FIG. 78. The system may, for example,e.g., find no pre-built 3D base model to associate with the set ofshelves 2102, or, for that matter, the armchair 1802. The system maythen build 3D base models from scratch or on the fly in an automatedprocess to associate with the set of shelves 2102 or the armchair 1802.The complexity of the shape of the image of the product may impactwhether the product model type may be defined, whether a pre-built 3Dbase model may be found to associate with the home design product, orwhether a 3D base model may be built from scratch or on the fly in anautomated process. For example, furniture may present more complexitythan a more box shaped appliance, such as a range.

As described above, the system may receive the graphical image 1600 ofthe range 1602 of FIG. 74 and may define the product model type of therange 1602 depicted in the picture 1600. Upon identifying the productmodel type of the range 1602, the system may retrieve one or morepre-built 3D base models that correspond to the defined or identifiedproduct model type of the range 1602. For example, the system mayretrieve a pre-built 3D base model such as the example pre-built 3D basemodel 2200 for the range 1602 (shown in FIG. 78). The system may alsobuild a non-complex 3D base model such as the 3D base model 2200 fromscratch or on the fly, depending on, e.g., the complexity of thegraphical image of the home design product, and the product model typeof the home design product.

In an implementation, the system may utilize the pre-built 3D base modelsuch as the model 2200 to build the 3D representation of the home designproduct, rather than building a 3D base model from scratch or on thefly.

For surfaces shown in the graphical image of the home design product,the system may perform automated image interpretation. The system mayevaluate the graphical image to define individual surfaces to extractout of the home design product image, based, for example, on thepre-built 3D base model. The system may use edge detection techniques todefine the individual surfaces to extract. The system may draw on avariety of information sources to define individual surfaces forextraction, including sources such as data from databases of knownattributes of product types and images, and adaptive learning usingneural networks that analyze prior user design behavior and selections,for example. The individual surfaces may include, top, front, side, andthe like. For example, for the graphical image 1600 of the range 1602,the system may detect edges to define surfaces for cropping images thatmay be applied as textures (here, cropped images) to portions of thepre-built 3D base model 2200 using texture mapping techniques such asstretching. The system may detect the surface 1604 as the front of therange 1602 of FIG. 74 that may correspond to a surface 2202 of the 3Dbase model 2200 of FIG. 78. Based on the product model type, the systemmay detect and identify “grill” and “panel” surfaces 1606, 1608 that maycorrespond to surfaces 2204, 2206 of the 3D base model 2200.

In applying edge detection to find, e.g., clipping areas for croppingimages in the graphical image, the system may detect edge detectiondetails that may be extraneous and that need not be captured as aseparate surface in a 3D base model. The system may ignore the edgedetection details when defining individual surfaces from the graphicalimage of the home design product to apply to the 3D base model. Forexample, on the range 1602 of the picture 1600, the system may detectvia edge detection the windows 1610, 1612 of the front of the range. Thesystem may ignore the windows as 1610, 1612 extraneous details ratherthan core features of the home design product and may instead define thefront surface 1604 as a entire surface for cropping. The system may cropthe entire front surface 1604 as one image that includes the windows aspart of the image. What is an extraneous detail may in general depend onthe product type of the home design product. For example, for the range,knobs and other instrumentation on the panel may be extraneous details.For a refrigerator, the door handles may be extraneous details, forexample.

The system may also draw on a variety of information sources toincorporate edge detection details for certain home design products,including sources such as data from databases of known attributes ofproduct types and images, and adaptive learning using neural networksthat analyze prior user design behavior and selections, for example. Forexample, for a sink, the system may detect the edges of the basin of thesink. In a 3D base model of a sink or a tub, the portion of the modelthat includes the basin of the sink or the inside of the tub may includerough edges at the bottom of the sink or the inside of the tub. Asdiscussed above, while the system may use tessellation techniques ofvarying complexity in 3D modeling to approximate rounded looking shapesso that a 3D base model does not include rough internal edges, thesystem may also, or instead use the texture mapping technique of warpingand may use texture warping effects to remove hard angles of the 3D basemodel, such as the edges of the basin of a sink. Warping a texture mayinclude applying a texture to fill in and smooth out rough internaledges of the 3D base model. In this way, for example warping techniquesmay create the effect of a basin.

For surfaces shown in the graphical image of the home design product,the system may crop images from the graphical image of the home designproduct and then may apply the cropped images to corresponding surfacesof a pre-built 3D base model. The texture mapping technique ofstretching may be used by the system to apply the textures (here,cropped images) to the 3D base model. For the graphical image 1600 ofthe range 1602 (FIG. 74), the system may apply cropped images from thesurfaces 1604, 1606, 1608 to corresponding surfaces 2202, 2204, 2206 ofthe 3D base model 2200 (FIG. 78) to form the 3D representation 2900 ofthe range 1602 in FIG. 96. The 3D representation 2900 includes a “front”surface 2902, a “grill” surface 2904, a “panel” surface 2906,” a rightside surface 2908, and a “top” surface 2910. The back and right sidesurfaces are not shown in the particular view of the 3D representation2900. The sides, the back and the top of the range 1602 are not shown inthe graphical image 1600 of the range 1602.

For surfaces not shown in the graphical image of the home designproduct, the system may perform automated image interpretation. Thesystem may automatically apply stored textures, such as proceduraltextures (described above), that may not be part of the home designproduct image to portions of the 3D base model for which the system maynot have found a corresponding surface in the graphical image of thehome design product. The system may retrieve the stored textures basedon scraped data from a website of a manufacturer of a retailer of thehome design product, such as folksonomy and taxonomy based data. Forexample, there may be scraped data for the range 1602 that indicatesthat the range 1602 has surfaces made of stainless steel. The system mayretrieve the stored textures based on other criteria such as data fromdatabases of known attributes of product types and images, and adaptivelearning using neural networks that analyze prior user design behaviorand selections, for example. For example, the system or a user of thesystem may have previously applied a stored texture to the same productor a product with the same or a similar product model type (e.g., aproduct model type sharing certain characteristics as defined by, e.g.,a taxonomy/folksonomy information), and the system may leverage thisknowledge to automatically apply a stored texture to the 3D base model.The system may automatically apply a stored texture using a texturemapping technique such as tiling, as described above, although othertechniques such as warping and stretching may be used.

The system may automatically extract textures directly from the homedesign product image as cropped images and may automatically apply thetextures to a 3D base model, Certain cropped images from the home designproduct image may also be used as approximate procedural textures andmapped together to apply to a surface or surfaces of a 3D base model.More generally, the cropped images may be shapes that may be tiled,stretched or warped on to the surfaces of a 3D base model. For tiling,the cropped image may typically be square. For the example of the range1602, the system may automatically extract a cropped image of the rangesuch as an image that shows stainless steel and may automatically tilethe cropped image onto the left side (not shown), right side 2208, back(not shown), and top 2210 surfaces of the 3D base model 2200.

The system may select a texture for extraction as a cropped image fromthe graphical image of the home design product based on imageinterpretation. The system may automatically select a texture forextraction as a cropped image based on scraped data from a website of amanufacturer of a retailer of the home design product, such asfolksonomy and taxonomy based data. For example, there may be scrapeddata for the range 1602 that indicates that the range 1602 has surfacesmade of stainless steel.

The system may select a texture for extraction as a cropped image fromthe graphical image of the home design product based on other criteriasuch as data from databases of known attributes of product types andimages, and adaptive learning using neural networks that analyze prioruser design behavior and selections, for example. For example, thesystem or a user of the system, may have previously extracted a textureas a cropped image for the same product or a product with the same or asimilar product model type (model type sharing certain characteristicsas defined by, e.g., a taxonomy/folksonomy information). The system mayknow the size of, and location on a product of, cropped imagespreviously extracted and may leverage this knowledge. The system mayautomatically apply an extracted texture using a texture mappingtechnique such as tiling, as described above, although other techniquessuch as warping and stretching may be used.

With any textural details and textures applied to the 3D base model 2200of FIG. 78, the system may use the resulting example 3D representation2900 (see FIG. 96) of the range 1602 shown in the graphical image 1600(see FIG. 74) to create a 2D drawing, or 2D representation of the range1602. The system may use a top to down view of the 3D representation2900 to create the 2D drawing. The 2D drawing 2500 of the range 1602 isshown in FIG. 92. In an implementation of the home design tool, the 2Ddrawing 2500 may be used, for example, in a 2D layout of the home designtool, and may be converted to the 3D representation 2900 when, forexample, a user switches from a 2D mode to a 3D mode in the home designtool.

The system may provide a user with an opportunity to verify whether a 3Drepresentation of a home design product is satisfactory and may providethe user with an opportunity to edit the 3D representation, or to builda new 3D representation. The system may, in some circumstances, ask auser to verify, edit, or build a new 3D representation of a home designproduct, or any combination of these. User verification may occur whenthe system scrapes a graphical image of a home design product from awebsite. A user may switch from a 2D mode to a 3D mode in animplementation of the home design tool and a 3D representation for a 3Dlayout may be presented to the user for verification and possibleediting. A user may be asked to verify a 3D representation, or may begiven an opportunity to verify a 3D representation, when the userbookmarks a home design product, as shown, e.g., in FIGS. 97-98. In anynumber of such situations, e.g., scraping, 3D mode, bookmarking, a usermay be provided with an interface that may allow the user to confirm oredit a 3D representation, a 3D base model, or textures and texturemapping, for example.

In FIG. 97, a screenshot of a webpage 3000 shows a bookmarking interfacewhere a user may be presented with, for example, 2D and 3Drepresentations of the range 1602. The graphical image 1600 of the range1602 of FIG. 74 is displayed on the webpage 3000 of FIG. 97. Asdescribed above, the system may receive a graphical image of a homedesign product, when, for example, a user bookmarks a home designproduct, and the system may automatically build 2D and 3Drepresentations of the home design product. The 2D representation or 2Ddrawing 2500 and the 3D representation 2900 of the range 1602 aredisplayed on the webpage 3000 of FIG. 97. Clicking on an “Edit 3DRepresentation” button may bring up an editing interface 3102 as shownon the webpage 3100 of FIG. 97. The editing interface 3102 may allows auser to select a particular surface, “Front”, “Grill”, “Panel”,“Sides/Back” of the 3D base model 2200 for editing on the left hand sideof the editing interface 3102. The user may select “Front” which mayplace the focus on the front surface 3110 of the 3D base model 2200 andmay allow the user to work on and edit images to be applied to the frontsurface 3110, or may allow the user to verify that the front of theimage of the range 1602 was cropped properly.

The system may provide a user with numerous editing options. Forexample, for a graphical image of a home design product such as therange 1602, as shown in FIG. 98, the system may present a user with auser with a home design product image such as image 1600 and may allow auser to move boundaries of shapes, such as the boundaries 3104, 3106 ofthe rectangular shape defining the front of the range 1602 on the image1600. The shapes, or defined surfaces may be used to crop images forapplication onto surfaces of a 3D base model such as the 3D base model2200 shown in FIG. 98. For example, the system may have cropped theimage 3108 of the front of the range 1602 and applied the cropped imageusing a stretching texture mapping technique to the front surface 3110of the 3D base model 2200. A user may move the boundaries defining thesurface to better define the edges of the surface for selection as acropped image to be applied to the front surface 3110 of the 3D basemodel 2200.

In an implementation, the system may allow a user to define new surfaceson a home design product image or a 3D base model. The system may allowa user to design or edit her own 3D base model using, for example, basicdrawing tools such as rectangles. The system may allow a user to applytextures to an existing 3D base model or to her own designed or edited3D base model.

The system may allow a user to edit surfaces not shown in a graphicalimage of a home design product. The system may allow a user to draw aboundary around a portion of the home design product image and may allowthe user to apply the cropped image as a texture using texture mappingtechniques such as tiling, warping, or stretching, to surfaces of a 3Dbase model not shown in the graphical image. The system may allow a userto confirm, select, or edit a stored texture, such as a proceduraltexture for the system to apply to surfaces of the 3D base model notshown in the graphical image.

For example, referring to the editing interface 3102, the user mayselect “Sides/Back”, which may place the focus of the interface 3102 onthe surfaces of the 3D base model 2200 (e.g., sides and back surfacessuch as the right surface 3112) that are not shown in the graphicalimage 1600 of the range 1602. The system may permit the user to choose atexture from the existing image, or a stored texture such as aprocedural texture, to apply to the sides and back surfaces of the 3Dbase model 2200. The system may not have applied a texture to surfacesof the 3D base model not shown in the graphical image, so that when auser selects “Sides/Back” in the editing interface 3102, the system maypresent the user with different options for textures to apply to the 3Dbase model 2200 surfaces. The system may suggest a stainless steelstored texture that the system retrieved by, e.g., learning from scrapeddata that the range 1602 has stainless steel surfaces. The system mayevaluate the graphical image to find a texture on the image that mostclosely approximates, e.g., the color of stainless steel, and presentthe texture to the user, for example.

FIG. 99 is a flow chart of an example process 3200 for use in building athree-dimensional representation of a home design product. A graphicalimage of a home design product is processed (3202). A three-dimensional(3D) base model is associated with the graphical image (3204). A textureis applied to the 3D base model to build the three-dimensionalrepresentation of the home design product (3206).

While the systems, methods, and computer program products describedabove have been directed to an implementation of the home design toolfor design and layout of rooms for a house, the tool is not limited tohome designs, as such, other implementations are possible. For example,the home design tool could be used to generate a layout of an officebuilding or other commercial space. In addition, the home design toolcould be used to generate layouts of garden or other landscapingfeatures.

In using the terms “2D,” “two-dimensional,” “3D,” and“three-dimensional,” these terms are understood to the virtual world,i.e., virtual 3D, virtual 2D, etc., rather than the physical world.

In using the term “may,” it is understood to mean “could, but notnecessarily must.”

In using the “set” as in “a set of elements,” it is understood that aset may include one or more elements.

The processes described herein are not limited to use with anyparticular hardware, software, or programming language; they may findapplicability in any computing or processing environment and with anytype of machine that is capable of running machine-readableinstructions. All or part of the processes can be implemented in digitalelectronic circuitry, or in computer hardware, firmware, software, or incombinations thereof.

All or part of the processes can be implemented as a computer programproduct, e.g., a computer program tangibly embodied in one or moreinformation carriers, e.g., in one or more machine-readable storagemedia or in a propagated signal, for execution by, or to control theoperation of, data processing apparatus, e.g., a programmable processor;a computer, or multiple computers. A computer program can be written inany form of programming language, including compiled or interpretedlanguages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, or other unitsuitable for use in a computing environment. A computer program can bedeployed to be executed on one computer or on multiple computers at onesite or distributed across multiple sites and interconnected by acommunication network.

Actions associated with the processes can be performed by one or moreprogrammable processors executing one or more computer programs toperform the functions of the processes. The actions can also beperformed by, and the processes can be implemented as, special purposelogic circuitry, e.g., an FPGA (field programmable gate array) or anASIC (application-specific integrated circuit). Modules can refer toportions of the computer program and/or the processor/special circuitrythat implements that functionality.

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally,one or more processors will receive instructions and data from aread-only memory or a random access memory or both. The essentialelements of a computer are one or more processors for executinginstructions and one or more memory devices for storing instructions anddata. Generally, a computer will also include, or be operatively coupledto receive data from or transfer data to, or both, one or more massstorage devices for storing data, e.g., magnetic, magneto-optical disks,or optical disks. Information carriers suitable for embodying computerprogram instructions and data include all forms of non-volatile memory,including by way of example semiconductor memory devices, e.g., EPROM,EEPROM, and flash memory devices; magnetic disks, e.g., internal harddisks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROMdisks. The processor and the memory can be supplemented by, orincorporated in special purpose logic circuitry.

An example of one such type of computer is shown in FIG. 100, whichshows a block diagram of a programmable processing system (system) 511suitable for implementing or performing the apparatus or methodsdescribed herein. The system 511 includes one or more processors 520, arandom access memory (RAM) 521, a program memory 522 (for example, awriteable read-only memory memo (ROM) such as a flash ROM), a hard drivecontroller 523, and an input/output (I/O) controller 524 coupled by aprocessor (CPU) bus 525. The system 511 can be preprogrammed, in ROM,for example, or it can be programmed (and reprogrammed) by loading aprogram from another source (for example, from a floppy disk, a CD-ROM,or another computer).

The hard drive controller 523 is coupled to a hard disk 130 suitable forstoring executable computer programs, including programs embodying thepresent methods, and data including storage. The I/O controller 524 iscoupled by an I/O bus 526 to an I/O interface 527. The I/O interface 527receives and transmits data in analog or digital form over communicationlinks such as a serial link, local area network, wireless link, andparallel link.

To provide for interaction with a user, the techniques described hereincan be implemented on a computer having a display device, e.g., a CRT(cathode ray tube) or LCD (liquid crystal display) monitor, fordisplaying information to the user and a keyboard and a pointing device,e.g., a mouse or a trackball, by which the user can provide input to thecomputer (e.g., interact with a user interface element, for example, byclicking a button on such a pointing device). Other kinds of devices canbe used to provide for interaction with a user as well; for example,feedback provided to the user can be any form of sensory feedback, e.g.,visual feedback, auditory feedback, or tactile feedback; and input fromthe user can be received in any form, including acoustic, speech, ortactile input.

The techniques described herein can be implemented in a distributedcomputing system that includes a back-end component, e.g., as a dataserver, and/or a middleware component, e.g., an application server,and/or a front-end component, e.g., a client computer having a graphicaluser interface and/or a Web browser through which a user can interactwith an implementation of the invention, or any combination of suchback-end, middleware, or front-end components. The components of thesystem can be interconnected by any form or medium of digital datacommunication, e.g., a communication network. Examples of communicationnetworks include a local area network (“LAN”) and a wide area network(“WAN”), e.g., the Internet, and include both wired and wirelessnetworks.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interact overa communication network. The relationship of client and server arises byvirtue of computer programs running on the respective computers andhaving a client-server relationship to each other.

Actions associated with the processes can be rearranged and/or one ormore such action can be omitted to achieve the same, or similar, resultsto those described herein.

Elements of different implementations may be combined to formimplementations not specifically described herein.

Numerous uses of and departures from the specific system and processesdisclosed herein may be made without departing from the inventiveconcepts. Consequently, the invention is to be construed as embracingeach and every novel feature and novel combination of features disclosedherein and limited only by the spirit and scope of the appended claims.

What is claimed is:
 1. A computer-implemented method for use in buildinga three-dimensional representation of a home design product, the methodcomprising: processing a graphical image of the home design product;associating a three-dimensional base model with the graphical image; andapplying a texture to the three-dimensional base model to build thethree-dimensional representation of the home design product, whereinprocessing the graphical image and associating the three-dimensionalbase model with the graphical image comprises building thethree-dimensional base model, wherein building the three-dimensionalbase model further comprises: applying edge detection to the graphicalimage; determining main object planes based on the graphical image;forming a bounding box from the main object planes, wherein the boundingbox comprises an outer boundary for the three-dimensional base model;and assigning a feature level plane within the bounding box.
 2. Thecomputer-implemented method of claim 1, further comprising generating avisual bookmark of the home design product.
 3. The computer-implementedmethod of claim 1, further comprising: scraping information regardingthe home design product from a website; wherein the informationcomprises the graphical image.
 4. The computer-implemented method ofclaim 1, further comprising: receiving the graphical image; wherein thethree-dimensional base model is built prior to the graphical image beingreceived.
 5. The computer-implemented method of claim 1, furthercomprising selectively ignoring a feature detail of the graphical imagefound while applying edge detection as an extraneous detail.
 6. Thecomputer-implemented method of claim 1, further comprising: presentingthe three-dimensional representation to a user; and requesting that theuser verify the three-dimensional representation.
 7. Thecomputer-implemented method of claim 1, wherein building thethree-dimensional base model comprises determining perspectiveinformation for the graphical image.
 8. The computer-implemented methodof claim 1, wherein building the three-dimensional base model furthercomprises: assigning dimensional information to the graphical image;wherein the dimensional information comprises at least one of overalldimensions for the home design product, or dimensions for a feature ofthe home design product.
 9. The computer-implemented method of claim 8,wherein building the three-dimensional base model further comprisessetting a relative scale for the three-dimensional base model using thedimensional information.
 10. The computer-implemented method of claim 1,wherein building the three-dimensional base model further comprisesapplying dimensional information to the main object planes to define oneor more distances between the main object planes.
 11. Thecomputer-implemented method of claim 1, wherein the texture comprises astored texture.
 12. The computer-implemented method of claim 1, furthercomprising: applying a textural detail to the three-dimensional basemodel to build the three-dimensional representation; wherein thetextural detail comprises a bump map.
 13. The computer-implementedmethod of claim 1, further comprising: presenting the three-dimensionalbase model to a user; and requesting that the user verify thethree-dimensional base model.
 14. The computer-implemented method ofclaim 1, further comprising: providing an interface to a user to allow auser to edit at least one of the three-dimensional representation or thethree-dimensional model.
 15. The computer-implemented method of claim 1,further comprising: building a two-dimensional drawing of the homedesign product using the three-dimensional representation.
 16. Acomputer-implemented method for use in building a three-dimensionalrepresentation of a home design product, the method comprising:processing a graphical image of the home design product; associating athree-dimensional base model with the graphical image; and applying atexture to the three-dimensional base model to build thethree-dimensional representation of the home design product, whereinprocessing the graphical image and associating the three-dimensionalbase model with the graphical image comprises building thethree-dimensional base model, wherein building the three-dimensionalbase model further comprises: applying edge detection to the graphicalimage; defining a profile on a feature level plane; wherein the profilecorresponds to a feature of the graphical image.
 17. Thecomputer-implemented method of claim 16, wherein building thethree-dimensional base model further comprises: retrieving the profilefrom a profile database; wherein the profile is parametric; andadjusting the profile to match a shape of the feature of the graphicalimage; wherein the feature of the graphical image intersects the featurelevel plane.
 18. The computer-implemented method of claim 16, whereinbuilding the three-dimensional base model further comprises: drawing theprofile to match a shape of the feature of the graphical image; whereinthe feature of the graphical image intersects the feature level plane.19. The computer-implemented method of claim 16, wherein building thethree-dimensional base model further comprises: applying a solidmodeling technique to the profile to define a feature in thethree-dimensional base model.
 20. The computer-implemented method ofclaim 19, wherein applying the solid modeling technique comprises:extruding the profile to build a three-dimensional model of the feature.21. The computer-implemented method of claim 20, wherein building thethree-dimensional base model further comprises: warping a surface of thethree-dimensional model of the feature.
 22. The computer-implementedmethod of claim 16, wherein building the three-dimensional base modelfurther comprises: mirroring the profile in the feature level plane. 23.The computer-implemented method of claim 16, wherein building thethree-dimensional base model further comprises: determining a secondfeature level plane for the feature of the graphical image; defining asecond profile on the second feature level plane; wherein the secondprofile corresponds to the feature of the graphical image; evaluatingthe profile and the second profile; selecting a best profile of theprofile and the second profile; and applying a solid modeling techniqueto the best profile to define the feature in the three-dimensional basemodel.
 24. A system comprising: an apparatus, the apparatus comprising:memory configured to store instructions for execution; and one or moreprocessing devices configured to execute the instructions, theinstructions for causing the one or more processing devices to: processa graphical image of the home design product; associate athree-dimensional base model with the graphical image; and apply atexture to the three-dimensional base model to build a three-dimensionalrepresentation of the home design product, wherein processing thegraphical image and associating the three-dimensional base model withthe graphical image comprises building the three-dimensional base model,wherein building the three-dimensional base model further comprises:applying edge detection to the graphical image; determining main objectplanes based on the graphical image; forming a bounding box from themain object planes, wherein the bounding box comprises an outer boundaryfor the three-dimensional base model; and assigning a feature levelplane within the bounding box.